Translation and psychometric evaluation of composite feedback-seeking behavior questionnaire among Iranian medical residents.

BACKGROUND: Proactively seeking feedback from clinical supervisors, peers or other healthcare professionals is a valuable mechanism for residents to obtain useful information about and improve their performance in clinical settings. Given the scant studies investigating the limited aspects of psychometrics properties of the feedback-seeking instruments in medical education, this study aimed to translate the feedback-seeking behavior scales (frequency of feedback-seeking, motives of feedback-seeking, and promotion of feedback-seeking by supervisors) into Persian and evaluate the psychometric properties of the composite questionnaire among medical residents at Tehran University of Medical Sciences in Iran. METHODS: In this cross-sectional study, feedback-seeking behavior scales were translated through the forward-backward method, and its face validity and content validity were assessed by 10 medical residents and 18 experts. The test-retest reliability was evaluated by administering the questionnaire to 20 medical residents on two testing occasions. A convenience sample of 548 residents completed the questionnaire. Construct validity was examined by exploratory factor analysis and confirmatory factor analysis and concurrent validity was determined by Pearson's correlation coefficient. RESULTS: Content validity assessment showed that the CVR (0.66 to 0.99) and CVI (0.82 to 0.99) values for items and S-CVI values (0.88 to 0.99) for scales were satisfactory. The exploratory and confirmatory factor analysis demonstrated that the models were confirmed with eight items and two factors (explaining 70.98% of the total variance) for the frequency of feedback-seeking scale, with 16 items and four factors (explaining 73.22% of the total variance) for the motives of feedback seeking scale and with four items and one factor (explaining 69.46% of the total variance) for promotion of feedback-seeking by supervisors. AVE values greater than 0.5 and discriminant validity correlations significantly less than 1.0 demonstrated that the total scores of the composite feedback-seeking behavior questionnaire had a favorable fit and the questions could fit their respective factors, and the latent variables were distinct. We found positive and significant correlations between the three scales and their subscales. CONCLUSION: The results of the present study supported the validity and reliability of the Persian composite feedback-seeking behavior questionnaire for assessing feedback-seeking behaviors in medical residents. Applying the questionnaire in residency programs may enhance the quality of clinical education.

Clearing the confusion about post-accreditation monitoring, meta-evaluation and meta-accreditation.

We have recently published the experience of the accreditation body of undergraduate medical education in Iran on developing and validating standards based on the WFME framework (Gandomkar et al., BMC Med Educ 23:379, 2023). Agabagheri et al. extended our work and proposed a blueprint for post-accreditation monitoring based on their experience in developing an official guide in their Matters Arising (Aghabagheri et al., BMC Med Educ). The authors have used post-accreditation monitoring as a process of monitoring and controlling accreditation activities, procedures often referred to as meta-evaluation or meta-accreditation (depending on the objectives of evaluation) in the literature. On the contrary, post-accreditation monitoring alludes to the process of continuous quality improvement of educational programs after accreditation. We would like to make clarifications between post-accreditation monitoring, meta-evaluation and meta-accreditation which have been used interchangeably in their paper. Considering the emerging interests in scholarship and non-scholarship activities and reports in undergraduate medical education accreditation, this clarification provides a better understanding of the roles of these crucial concepts in the accreditation process.

The potential of structured learning diaries for combining the development and assessment of self-regulated learning.

Structured Self-Regulated Learning (SSRL) diaries have the potential for combining the development and assessment of a student's SRL processes over time. The aim of this study was to evaluate the extent to which an SSRL diary can develop SRL and provide a reliable longitudinal assessment of SRL development in academically low-achieving undergraduate medical students. We conducted a quasi-experimental study with low-achieving medical students at Tehran University of Medical Sciences. The intervention was a weekly SSRL diary, with 21 items in two parts (before and after studying) that was integrated with weekly explicit SRL training. A repeated measures ANOVA was performed to assess the participants' SRL development across time. We conducted generalizability theory analysis in two designs; the first was to assess the reliability of the total diary score in assessing low-achieving medical students' SRL and the second was to assess the efficacy of the four-week intervention results in improving the low-achieving medical students' SRL. Each participant (n = 20) completed four SSRL diaries. There were significant positive changes during the intervention in the students' measures of total SRL, time estimation of study, time spent on study, concentration, elaboration, organization, self-monitoring, and self-evaluation. The absolute and relative generalizability coefficients for the first design, which indicates the reliability of the students' SRL scores, were 0.71 and 0.77. The absolute and relative generalizability coefficients for the second design, which presents the reliability of the improvement of students' SRL across time were 0.79 and 0.87. The results of this study suggest that SSRL diaries combined with explicit training can provide an effective and reliable method for combining the development and assessment of SRL in low-achieving medical students' SRL over time.

Developing and validating a national set of standards for undergraduate medical education using the WFME framework: the experience of an accreditation system in Iran.

BACKGROUND: Defining standards is the first step toward quality assurance and improvement of educational programs. This study aimed at developing and validating a set of national standards for the Undergraduate Medical Education (UME) program through an accreditation system in Iran using the World Federation for Medical Education (WFME) framework. METHODS: The first draft of standards was prepared through consultative workshops with the participation of different UME program stakeholders. Subsequently, standards were sent to medical schools and UME directors were asked to complete a web-based survey. The content validity index at the item level (I-CVI) was computed using criteria including clarity, relevance, optimization and evaluability for each standard. Afterward, a full-day consultative workshop was held and a wide range of UME stakeholders across the country (n = 150) discussed the survey results and made corrections to standards. RESULTS: Analysis of survey results showed that relevance criteria had the best CVI as only 15 (13%) standards demonstrated CVI < 0.78. More than two-thirds (71%) and a half (55%) of standards showed CVI < 0.78 for optimization and evaluability criteria. The final set of UME national standards was structured in 9 areas, 24 sub-areas, 82 basic and 40 quality development standards, and 84 annotations. CONCLUSIONS: We developed and validated national standards as a framework to ensure the quality of UME training with input from UME stakeholders. We used WFME standards as a benchmark while addressing local requirements. The standards and participatory approach to developing standards may guide relevant institutions.

Enhancing Medical Students' Knowledge and Performance in Otolaryngology Rotation through Combining Microlearning and Task-Based Learning Strategies.

Microlearning is recommended to be implemented within the context of a wider teaching-learning ecosystem, especially in real working environments. Task-based learning is used in clinical education setting. This study aims at assessing the effect of an integrated approach of microlearning with task-based learning on medical students' knowledge and performance in Ear, Nose and Throat clerkship rotation. A total of 59 final-year medical students participated in this quasi-experimental study which included two control groups (routine teaching and task-based learning methods) and one intervention group (combined microlearning and task-based learning method). Pre- and post-tests of students' knowledge and performance were assessed through a multiple-choice question test and a Direct Observation Procedural Skills (DOPS) instrument, respectively. Performing Analysis of Covariance for knowledge post-test scores among three groups revealed significant differences (F = 3.423, p-value = 0.040) and the intervention group had the highest score. Analyzing DOPS results showed that the intervention group achieved significantly higher scores compared to the control ones for all the expected tasks (p-values = 0.001). The findings of the present study show that the combined strategy of microlearning with task-based learning is an effective clinical teaching method for enhancing medical students' knowledge and performance in a real working environment.

Developing low-achieving medical students' self-regulated learning using a combined learning diary and explicit training intervention.

INTRODUCTION: The development of self-regulated learning (SRL) is an essential educational component of remediation for low-achieving students. The aim of this study was to design, implement, and evaluate a longitudinal SRL intervention combining both a structured learning diary and explicit SRL training in a cohort of low-achieving undergraduate medical students. MATERIALS AND METHODS: A mixed methods quasi-experimental study was conducted, with a pretest-posttest study in the intervention group and comparison of the GPA and course grade of the intervention group with a historical comparison group. A questionnaire and focus group explored the participants' perceptions about the intervention. RESULTS: The SRL scores (total and rehearsal, organization, critical thinking, metacognitive regulation, time management and environment management) and course grade of participants were significantly improved in the intervention group. The course grade of participants was significantly higher than the comparison group but the GPA was not significantly different. Overall, the participants were positive about the intervention. CONCLUSIONS: This study was the first in medical education to evaluate the effectiveness and user acceptability of an SRL intervention that combined a structured learning diary and explicit SRL training in low-achieving medical students. Further research is recommended in different contexts and with larger number of students.

Peer-review of teaching materials in Canadian and Australian universities: A content analysis.

BACKGROUND: Peer-review of teaching materials (PRTM) has been considered a rigorous method to evaluate teaching performance to overcome the student evaluation's psychometric limitations and capture the complexity and multidimensionality of teaching. The current study aims to analyze the PRTM practices in Canadian and Australian universities in their faculty evaluation system. MATERIALS AND METHODS: This is a qualitative content analysis study in which all websites of Canadian and Australian universities (n = 46) were searched based on the experts> opinion. Data related to PRTM were extracted and analyzed employing an integrative content analysis, incorporating both inductive and deductive elements iteratively. Data were coded and then organized into subcategories and categories using a predetermined framework including the major design elements of a PRTM system. The number of universities for each subcategory was calculated. RESULTS: A total of 21 universities provided information on PRTM on their websites. The main features of PRTM programs were organized under the seven major design elements. Universities applied PRTM mostly (n = 11) as a summative evaluation. Between half to two-thirds of the universities did not provide information regarding the identification of the reviewers and candidates, preparation of reviewers, and logistics (how often and when) of the PRTM. Almost all universities (n = 20) defined the criteria for review in terms of teaching philosophy (n = 20), teaching activities (n = 20), teaching effectiveness (n = 19), educational leadership (n = 18), teaching scholarship (n = 17), and professional development (n = 14). CONCLUSION: The major design elements of PRTM, categories and subcategories offered in the current study provide a practical framework to design and implement a comprehensive and detailed PRTM system in the academic setting.

Faculty Development in the COVID-19 Era: A Rapid Systematic Review.

Background: The coronavirus-2019 (COVID-19) pandemic is a worldwide evolving situation that has resulted in rapid adaptations of faculty development interventions in medical education. The present rapid systematic review aims to provide a narrative synthesis of the evidence concerning focus, intervention type, instructional methods, duration, and the evaluations results of the faculty development interventions in the medical education area in response to the COVID-19 pandemic. Methods: This was a systematic study conducted on 5 databases (Medline/PubMed, EMBASE, Web of Science, ERIC, and Scopus) from December 2019 to November 2021. We used specific keywords such as faculty development, COVID-19, and medical education on electronic databases. To find additional relevant studies, we conducted a forward and backward searching strategy by checking the reference lists and citations of the included articles. Studies reporting the educational faculty development interventions in medical education during the COVID-19 era and those articles published in English were included. Titles, abstracts, and full texts were screened and the data were extracted by 2 authors. Results: Ten articles were included, most of which were focused on the improvement of online teaching and learning skills. The findings are organized into 2 main categories, including a description of the interventions, and the special aspects of the studies in response to the COVID-19 pandemic. Conclusion: This review explores the evidence concerning faculty development programs in the medical education area in response to the COVID-19 pandemic. These interventions may develop individual abilities and organizational capacities of health professions educators to enable them to sustain academic vitality and cope with the pandemic crisis. Also a rapid movement to online faculty developments, which will likely continue after the pandemic, was argued and it is required to direct the adaptations and innovations of educational developments to an organized structure in the future.

Leadership Behaviors in Health Care Action Teams: A Systematized Review.

Background: Effective leadership is critical for the performance of health care teams and their intended outcomes for patient care. Given that team leadership is a modifiable and teachable skill, there is a need for a better understanding of this multidimensional behavior to inform future leadership training for health care action (HCA) teams. This systematized review identifies reported observed leadership behaviors in HCA teams, defined as interdisciplinary teams which complete vital tasks in complex, time-pressured, and dynamic situations. Methods: We searched CINAHL, MEDLINE, Scopus, PsycINFO, and Web of Science for peer-reviewed, English language articles using single and combinations of keywords including leadership, health care action team, and teamwork, individually. We included articles published until June 2021 without any specific beginning date. Results: From 242 records, 13 articles were included in the review. We categorized our findings of team leadership behaviors in HCAs based on an existing framework of three dimensions: transition processes, action processes, and interpersonal skills. The most-reported behaviors for transition processes were encouraging team members' input, (re)assessing the team's situation, and confirming team members' understandings. The action processes dimension consisted of behaviors that included monitoring the progress of the patient, managing resources, asking for help when needed, coaching/supervising, and assisting team members as needed. Finally, closed-loop communication and facilitating team members speaking up behaviors were categorized as interpersonal skills. Conclusion: Although team leadership has been an area of focus in the field of health professions education, little attention has been paid to identifying the observable behaviors of effective team leaders in an HCA team. The study identified several new essential team leadership behaviors that had not been previously described, including seeking feedback, shared decision making, and aspects of interpersonal communication. The findings can inform educators in planning and implementing strategies to enhance HCA team leadership training, with the ultimate potential to improve health care.

Expectations for PhDs in health professions education: an international EPA-framed, modified Delphi study.

Health professions education (HPE) has matured into field of study that employs and produces specialized educational scholars. Many academic institutions employ such scholars to support development and innovation in education. Simultaneously, the possibilities to obtain a PhD degree in HPE are expanding. Clarity in the expectations that such a degree brings along can be useful for scholars, employers and institutions. Anticipating that the views of what a PhD in HPE is or should be can vary between institutions, cultures and countries, we conducted an international Delphi study to identify EPAs for HPE PhDs. We used a framework of 24 EPAs resulted from a national consensus study in Iran as input to seek more generalized validity and a wider consensus of reasonably suitable activities. A three-round modified Delphi study was conducted with participation of 15 international experts. Final framework consisted of 17 relevant EPAs with a justification, specifications and limitations, and risks in case of failure per EPA (all had overall CVI > 0.8 for adequacy of description). Eleven EPAs were considered by the majority to be core to PhD training, 16 to be desirable for HPE PhDs as a capability to carry out without supervision, but the majority would trust current HPE PhDs to carry out only 5 of the 17 without supervision. The EPAs identified in this study arguably reflect the expectations of educators with a PhD degree in HPE. The framework has the potential to advance HPE training across countries and to inform personal development and employment decisions.

Developing entrustable professional activities for doctoral graduates in health professions education: obtaining a national consensus in Iran.

BACKGROUND: The number of doctoral programs in health professions education (HPE) is expanding. Entrustable professional activities (EPAs) can be a mechanism to define the expected activities of the HPE doctorate to inform training and assessment processes. The purpose of this study was to develop and reach a consensus on EPAs for HPE doctoral graduates. METHODS: We used a modified Nominal Group Technique (NGT) to elicit EPA titles followed by two rounds of a modified Delphi survey to seek consensus on the EPAs among groups of experts (HPE doctoral graduates and Board of HPE Examiners members) at the national level in Iran between July 2019 and July 2020. RESULTS: A total number of 92 initial EPA titles, which emerged from brainstorming in the NGT meeting, was reduced to 27 titles during the clarification process. The final EPA framework consisted of 24 EPA titles with descriptions, arranged in three categories: Research and scholarship (6 EPAs), Educational development (11 EPAs) and Educational management (7 EPAs). All final EPAs scored ≥80% agreement at the national level. CONCLUSIONS: The proposed EPAs framework can be used to improve the HPE doctorate training and to inform employment decisions. A future international consensus procedure could use these EPA outcomes as a starting point.

Identifying Situational Awareness Behaviors in Trauma Teams; a Nominal Group Technique Study.

Introduction: Situational awareness (SA), as a nontechnical human factor, is critical to the success of a trauma team. This study aimed to identify representatives of behaviors supporting (desirable) and diminishing (undesirable) SA for trauma teams while performing the initial assessment of multi-trauma patients. Methods: This Nominal Group Technique Study was conducted on twenty attending physicians from various specialties affiliated with Tehran University of Medical Sciences, who were invited to a nominal group technique meeting in 2020. Participants were asked to write down their proposed behaviors in silence. Subsequently, each participant shared their list with the group in a round-robin format, and clarifications were made through discussion. After categorizing the ideas, we asked participants to rate each behavior's importance on a five-point Likert scale. The consensus was defined as ≥70% agreement on a rating of 4 and 5. Results: The final SA behaviors for the trauma team consisted of 29 (22 desirable and 7 undesirable) behaviors arranged in seven dimensions: resource allocation, anticipate and plan, avoid fixation errors, call for help if needed, prioritize attention, reassess patient, and shared mental model. The most important desirable and undesirable behaviors were identified in resource allocation (n=8) and avoid fixation errors (n=7) dimensions, respectively. Resource allocation behaviors consist of 'checking necessary equipment', 'allocating an alternative person(s) to do the required task if needed', 'assigning tasks to the right person(s)', and 'Addressing each team member with a requested task'. Avoid fixation errors behaviors were 'insisting on performing the procedure', 'making decisions without considering all available information', and 'emphasizing others' expertise in the diagnostic process'. Conclusion: The proposed team SA behaviors may be used in assessing the trauma team performance and training program to promote trauma team SA.

Residency Education Reform Program in Department of Anesthesiology and Critical Care: An Academic Reform Model.

BACKGROUND: Reform in medical education is a basic process in every academic department, especially in residency programs. OBJECTIVES: This study was designed to assess the indices of education and research as part of the Medical Education Reform program (MERP) in the Department of Anesthesiology and Critical Care (DACC), Shahid Beheshti University of Medical Sciences (SBMU) for four years. METHODS: MERP in DACC, SBMU was designed and implemented as a modern academic reform model; different outcome measures in education and research were assessed to demonstrate the effects of the reform plan in academic improvements. RESULTS: there were significant improvements regarding education indices (i.e., teaching methods, passing comprehensive exams, mentorship, assessment methods, faculty development, professionalism in medical education, integration in education, and crisis management) and research indices (targeted research activities, innovation in research approaches, increasing the impact of research). CONCLUSIONS: Based on the experiences of DACC, SBMU regarding clinical anesthesiology residency, reform could be achieved using painstaking plans and continuous efforts with tangible documented outcomes. Often, the management period is not durable, and these reforms require meticulous care to sustain.

Avoid 'running before we can walk' in medical education research: The importance of design and development research.

Design and development research has the aim of understanding the feasibility and acceptability of implementing early-stage pilot research before interventions are fully implemented and evaluated for their impact. Increasing the use of design and development research in medical education research requires greater awareness of its importance by all stakeholders, the use of iterative research methodologies, such as educational design research, and the application of modified existing frameworks for healthcare feasibility studies.

The challenge of understanding, evaluating and providing feedback on regulation during group learning / O desafio em entender, avaliar e fornecer feedback em regulação durante aprendizagem em grupo

Learning in groups is commonly used in academic and clinical health professions education (HPE). There is growing recognition that regulation during learning is essential for both the individual learner and group learning. The authors in this article propose a practical approach for understanding, evaluating and providing feedback on regulation during group learning. The approach is informed by previous studies conducted in other areas of education. Three varieties of regulation during group learning are discussed: individual, co-regulation and shared regulation. Each variety of regulation has a focus on three essential activities during group learning: task, social and motivation. Illustrative scenarios are presented to describe how the approach can be practically used in HPE. The specific and additional focus on regulation can enhance current approaches for providing feedback on group learning and the authors discuss recommendations for practical implementation and future research.

Aprendizagem em grupo é muito utilizada no ensino dos profissionais de saúde tanto na parte acadêmica quanto na parte clínica. Há um crescente reconhecimento de que a regulação durante aprendizagem é essencial para o indivíduo e em grupo. Os autores deste artigo propõem um modelo prático para entender, avaliar e fornecer feedback em regulação durante a aprendizagem em grupo. Esse modelo é baseado por estudos publicados em outras áreas de educação, sendo composto por três variações de regulação durante aprendizagem em grupo, a saber, individual, corregulação e regulação compartilhada. Cada variação de regulação foca em três atividades essenciais durante a aprendizagem em grupo: tarefa, social e motivacional. Cenários ilustrativos são apresentados para descrever como esse modelo pode ser utilizado em ensino em saúde. O foco específico e adicional em regulação pode melhorar práticas de feedback em aprendizagem em grupos e os autores discutem recomendações para implementações práticas assim como pesquisas futuras.

Psychometric evaluation of a questionnaire to evaluate organizational capacity development for faculty development programs.

BACKGROUND: Organizational capacity development is an important outcome of faculty development programs, but there is a lack of an appropriate instrument for its evaluation. AIMS AND OBJECTIVES: The aim of this study was to develop a questionnaire to evaluate the organizational capacity development for faculty development programs and to test its psychometrics. MATERIALS AND METHODS: The initial capacity development for faculty development questionnaire (CDQ-FD) of 26 items was developed based on a literature review and opinion of experts. Content validity ratio (CVR), content validity index (CVI), content validity index for items (I-CVI), and the content validity index for scales (S-CVI) were computed for content validity. Confirmatory factor analysis (CFA) and exploratory factor analysis (EFA) were performed for construct validation. RESULTS: The score for CVR, CVI, I-CVI, and S-CVI was 0.71, 0.83, 0.87, and 0.90, respectively. EFA resulted in a three-factor model with total variance extraction of 64%. Cronbach's alpha and Spearman Brown coefficient were investigated for reliability assessment. The Cronbach's alpha of overall scale was 0.8 and the test-retest reliability of the overall scale was 0.78. The final CDQ-FD contained 21 items and three categories. CONCLUSIONS: The CDQ-FD questionnaire appears to be a valid and reliable instrument for the evaluation of organizational capacity development for faculty development in the medical education.

Identification of capacity development indicators for faculty development programs: A nominal group technique study.

BACKGROUND: Although there have been many research studies of the effectiveness of faculty development in health profession education, the contribution of these programs to organizational development through capacity development has not been studied. Further understanding of capacity development requires appropriate indicators and no previous indicators for faculty development of health profession educators were identified. The aim of the study was to identify indicators of capacity development in the context of faculty development programs at Tehran University of medical sciences (TUMS). METHODS: A nominal group technique session was conducted with key informants from faculty development program providers to generate and prioritize a list of capacity development indicators. RESULTS: A list of 26 indicators was generated and five categories were identified: Development and innovation in teaching and learning process, Development and innovation in communication and collaboration at different levels, Development and sustaining faculty development programs, Development of educational leadership and management, Development in scholarship. CONCLUSIONS: Capacity development for faculty development interventions of health profession educators is a process of engagement within a wider system, including individual and collective action, and involves the socialization of the teachers into suitable roles through professional identity development and participation within the wider system.