Influence of tutor interventions and group process on medical students' engagement in problem-based learning.

INTRODUCTION: Student engagement is influenced by several variables, among which are the teaching styles employed by faculty. In problem-based learning (PBL), the role of faculty is to facilitate the learning of the tutorial group as they work through clinical problems. However, the influence of tutor intervention styles and group process on engagement of students in PBL tutorials is unclear. METHODS: The study was conducted with year 2 and 3 medical students (n = 176) in PBL tutorial groups at the end of an integrated PBL course. Students evaluated their behavioural, cognitive and emotional engagement in PBL tutorials using a pre-validated 11-item questionnaire. Students also filled in a modified version of a previously published tutor intervention profile (TIP) questionnaire. The modified TIP questionnaire represents three constructs (1) steering the learning process (6 items), (2) stimulating student autonomy (4 items) and (3) establishing relatedness with students (3 items). In addition, PBL group process was evaluated using a 5-item nominal scale: (1) tutorial atmosphere, (2) listening and information sharing, (3) group performance, (4) decision making and (5) reaction to leadership. RESULTS: Establishing sense of relatedness in the group by PBL tutors was the most important predictor of emotional engagement (F = 41.213, ΔR2 = 0.191, ß = 0.438, P = 0.000). On the other hand, steering the learning process was a significant predictor of behavioural engagement (F = 19.0, ΔR2=0.098, ß = 0.314, P = 0.000). However, stimulating student autonomy was not a significant predictor of student engagement in PBL tutorials. On the other hand, enhancing the group process in PBL tutorials significantly predicts student engagement with strong impact on emotional and cognitive engagement of students. CONCLUSIONS: Establishing the sense of relatedness in the group and steering the learning process by PBL tutors as well as improving PBL group process are significant predictors of student engagement in PBL tutorials with emotional and cognitive engagement being the most sensitive variables affected.

Measuring sociocultural engagement of health professions education students: a psychometric study.

BACKGROUND: Sociocultural engagement of students refers to broadening viewpoints and providing awareness of, and respect for, diverse backgrounds and perspectives. However, there are no existing validated instruments in the literature for measuring sociocultural engagement of health professions education (HPE) students. Therefore, the aim of this study is to develop and validate a questionnaire designed to assess sociocultural engagement among HPE students. METHODS: The study included undergraduate HPE students (n = 683) at Gulf Medical University. The initial version of the sociocultural engagement of students' questionnaire (SESQ) was developed after extensive literature review and guided by the Global Learning Qualifications Framework. We then tested the content validity of the questionnaire by using focus group discussion with subject experts (n = 16) and pilot testing with students (n = 20). We distributed the content-validated version of the SESQ (16 items) to undergraduate students in six HPE colleges. To examine the construct validity and construct reliability of the questionnaire, we conducted exploratory factor analysis, followed by confirmatory factor analysis. RESULTS: Confirmatory factor analysis supported the two-factor structure which consists of 13 items with good fitness indices (χ2 = 214.35, df = 61, χ 2/df = 3.51, CFI = 0.98, RMSEA = 0.06, SRMR = 0.025, and AIC = 208.00). The two factors were sociocultural interactions (8 items) and sociocultural adaptation (5 items). The construct reliability of the total questionnaire is 0.97 and the two factors were 0.93 and 0.92 for sociocultural interactions and sociocultural adaptation, respectively. In addition, there were significant weak correlations between both factors of sociocultural engagement scores and student satisfaction with the university experience (r = .19 for each, P = .01). CONCLUSIONS: The sociocultural engagement of students' questionnaire exhibits good evidence of construct validity and reliability. Further studies will be required to test the validity of this questionnaire in other contexts.

Student engagement in health professions education: AMEE Guide No. 152.

This guide aims to support our colleagues to have comprehensive understanding of student engagement in health professions education. Despite the universal agreement about the significance of student engagement, there is lack of uniformity in conceptualizing and operationalizing this emerging construct. We review the theoretical basis explaining student engagement from three main perspectives: behavioral, psychological, and socio-cultural. In addition, we propose a contemporary and comprehensive framework for the student engagement in higher education, which is applicable to health professions education contexts. Drawing from this framework, we explain the conceptualization of the construct and its preceding factors, mediators, dimensions, spheres, and outcomes of student engagement. The proposed framework introduces student 'engagement through partnerships' as a novel component compared with the existing models of student engagement in higher education. This way, we are proposing a mixed model that not only considers the student as a 'customer' but also as a 'partner' in education. Engagement of students through partnerships include four areas: (1) provision of the education program, (2) scholarly research, (3) governance and quality assurance, and (4) community activities. This guide will provide practical applications on how to improve student engagement in health professions education. Finally, we highlight the current gaps in areas of research in the student engagement literature and suggested plans for future directions.[Box: see text].

Measurement of student engagement in health professions education: a review of literature.

Student engagement is a complex multidimensional construct that has attained great interest in health professions education (HPE). Definition and conceptualization of student engagement is an important step that should drive the development of the instruments for its measurement. We have recently proposed a comprehensive framework for student engagement in HPE with a definition of engagement as student investment of time and energy in academic and non-academic experiences that include learning, teaching, research, governance, and community activities. The dimensions of student engagement in this framework included the cognitive, affective, behavioral, agentic, and socio-cultural. Guided by the student engagement framework, this non-systematic review aims to identify, critically appraise, and summarize the existing methods for measuring student engagement in HPE. Extrapolating from higher education literature, we attempted to link the theoretical perspectives of student engagement with the published methods of its measurement in HPE context. In addition, we have described the different methods of measuring student engagement including self-report surveys, real time measures, direct observation, interviews/focus groups, and the use of multiple instruments. The span of engagement dimensions measured by self-report surveys ranges from one to five dimensions. However, measurement of agentic and sociocultural dimensions of engagement in HPE is still limited and further research is required. We have also reflected on the existing methods of measuring engagement of students as active partners in HPE. The review also describes the advantages, limitations, and psychometric properties of each method for measuring student engagement. We ended the review with a guiding conclusion on how to develop and select an instrument for measuring student engagement in HPE. Finally, we addressed the gaps in the literature about measuring engagement of HPE students and future research plans.

Construct validity of a questionnaire for measuring student engagement in problem-based learning tutorials.

BACKGROUND: Student engagement is student investment of time and energy in academic and non-academic experiences that include learning, teaching, research, governance, and community activities. Although previous studies provided some evidence of measuring student engagement in PBL tutorials, there are no existing quantitative studies in which cognitive, behavioral, and emotional engagement of students in PBL tutorials is measured. Therefore, this study aims to develop and examine the construct validity of a questionnaire for measuring cognitive, behavioral, and emotional engagement of students in PBL tutorials. METHODS: A 15-item questionnaire was developed guided by a previously published conceptual framework of student engagement. Focus group discussion (n = 12) with medical education experts was then conducted and the questionnaire was piloted with medical students. The questionnaire was then distributed to year 2 and 3 medical students (n = 176) in problem-based tutorial groups at the end of an integrated course, where PBL is the main strategy of learning. The validity of the internal structure of the questionnaire was tested by confirmatory factor analysis using structural equation modeling assuming five different models. Predictive validity evidence of the questionnaire was studied by examining the correlations between students' engagement and academic achievement. RESULTS: Confirmatory factor analysis indicates a good fit between the measurement and structural model of an 11-item questionnaire composed of a three-factor structure: behavioral engagement (3 items), emotional engagement (4 items), and cognitive engagement (4 items). Models in which the three latent factors were considered semi-independent provided the best fit. The construct reliabilities of behavioral, cognitive, and emotional factors were 0.82, 0.82, and 0.76, respectively. We failed however to find significant relationships between academic achievement and engagement. CONCLUSIONS: We found a strong evidence to support the construct validity of a three-factor structure of student engagement in PBL tutorial questionnaire. Further studies are required to test the validity of this instrument in other educational settings. The predictive validity is another area needing further scrutiny.

Student engagement in undergraduate medical education: A scoping review.

INTRODUCTION: Student engagement is defined as behavioural, cognitive and emotional aspects of students' academic experience in teaching, learning and research through interacting with other students, faculty and community. Despite the growing interest in the field of student engagement, medical education research in this area is still fragmented. This scoping review aims to contribute to the understanding of measurements, drivers and outcomes of medical student engagement. METHODS: The authors searched MEDLINE, PubMed, ProQuest, SCOPUS, ERIC, Science Direct and EBESCO for English articles published from 1990 until October 2021. In addition, we hand-searched key medical education journals and references in recently published articles. Using specific selection criteria, two authors independently reviewed the articles for eligibility, followed by data extraction using both quantitative and qualitative analysis. RESULTS: Of the 2136 retrieved articles, 51 studies were selected for the review, and 94% of them were published in the past 8 years. The prevailing measures of student engagement failed to cover the multidimensionality of the construct with more focus on the behavioural dimension. Quantitative methods represented two thirds of the studies with a higher frequency of cross-sectional designs and using self-reports. The identified drivers of medical students' engagement are directed mainly to modifying the context of the learning environment. These factors increased student engagement by fostering relevance of learning, positive student relationships with peers and faculty, and enhancing student agency, and sense of competence. Cognitive engagement appears to be a positive predictor of academic achievement, but the relationship with other dimensions of engagement is controversial. CONCLUSIONS: Student engagement in undergraduate medical education is an important, yet under-researched construct. The research that does exist suggests engagement to be malleable, something that can be modified by different types of interventions taking into consideration the context of education and practice. Further research is required, however, to address the gaps identified in this review.

The effect of access to electronic resources during examination on medical and dental students scores in summative assessment: quasi experimental study.

BACKGROUND: Access to electronic (E) resources has become an indispensable requirement in medical education and practice. OBJECTIVE: Our objective was to assess the effect of E-resources access during examination on end-course-exam scores of medical and dental students. METHODS: A quasi-experimental study which included two cohorts of medical (n = 106 & 85) and three cohorts of dental students (n = 66, 64 and 69) who took end-course- exams. Each exam was composed of two- parts (Part I and II), that encompassed equal number of questions and duration. Access to E-resources was allowed in part-II only. Items Difficulty Index (DI), Discrimination Index, (DisI), Point Biserial, (PBS) and cognitive level were determined. RESULTS: The study included 390 students. The proportion of items at various levels of DI, DisI, and PBS and the average values for item DI, DisI in both parts of each exam were comparable. The average scores in part-II were significantly higher than part-I (P < 0.001, < 0.001 and 0.04) and lower-order cognitive-level items scores were higher in three exams (P < 0.0001, 0.0001, 0.0001). Higher- order cognitive level items scores were comparable between part I and II in all courses. The significant factor for change in marks were questions cognitive level and type of the course. CONCLUSION: Access to E-resources during examination does not make a significant difference in scores of higher-order cognitive level items. Question cognitive level and course type were the significant factors for the change in exam scores when accessing E-resources. Time-restricted E-resources accessed tests that examine higher cognitive level item had no significant academic integrity drawback.

Implications of introducing case based radiological images in anatomy on teaching, learning and assessment of medical students: a mixed-methods study.

BACKGROUND: Introducing radiological anatomy in the preclinical curriculum can increase the understanding of Anatomy. Regardless of the integration when teaching anatomy, it is essential to maintain oversight as to what and how much is being taught. In addition, the knowledge requirements for preclinical students should be considered. The purpose of this kind of integration is that the student should be able to apply the knowledge which can help them better understand anatomy and not to make the course more challenging. This study aimed to understand whether adding radiological images would increase the difficulty level of the questions. METHODS: We introduced radiological images, including X Rays, CT scans and MRIs, when teaching anatomy in the preclinical curriculum. A class of 99 students were tested using A-type MCQs (n = 84). All 84 questions were categorized on whether they were case-based with or without a radiological image. The item analysis of both groups of test questions was then compared based on their difficulty and discrimination index. A qualitative student perception regarding the inclusion of radiological images in anatomy was also measured using a questionnaire with a 5-point Likert scale. RESULTS: The results showed that the performance level of the students was similar when comparing the test questions in both groups. The item analysis of the MCQs in the two groups revealed that by integrating radiological images when teaching anatomy, the various parameters in both groups of test questions were in the same range. More than 80% of the students felt that radiological images facilitate the achievement of learning outcomes and help to apply their knowledge in clinical contexts. The study's findings reported that the rate of satisfaction by including radiological images when teaching anatomy is high. CONCLUSION: Recognition and interpretation of images are essential in an undergraduate medical program. Students found it helpful when radiological images were introduced to them when teaching anatomy. Since the students' performance in summative exams in both groups of questions was in the same range, the findings also point out that adding radiological images when teaching anatomy does not increase the difficulty of the subject.

A guide to best practice in faculty development for health professions schools: a qualitative analysis.

BACKGROUND: This is a practice guide for the evaluation tool specifically created to objectively evaluate longitudinal faculty development programs (FDP) using the "5×2 -D backward planning faculty development model". It was necessary to create this tool as existing evaluation methods are designed to evaluate linear faculty development models with a specific endpoint. This backward planning approach is a cyclical model without an endpoint, consisting of 5 dynamic steps that are flexible and interchangeable, therefore can be a base for an evaluation tool that is objective and takes into account all the domains of the FDP in contrast to the existing, traditional, linear evaluation tools which focus on individual aspects of the program. The developed tool will target evaluation of longitudinal faculty development programs regardless of how they were planned. METHODOLOGY: Deductive qualitative grounded theory approach was used. Evaluation questions were generated and tailored based on the 5 × 2-D model followed by 2 Delphi rounds to finalize them. Based on the finalized evaluation questions from the results of the Delphi rounds, two online focus group discussions (FGDs) were conducted to deduce the indicators, data sources and data collection method. RESULTS: Based on the suggested additions, the authors added 1 new question to domains B, with a total of 42 modifications, such as wording changes or discarding or merging questions. Some domains received no comments, therefore, were not included in round 2. For each evaluation question, authors generated indicators, data sources and data collection methods during the FGD. CONCLUSION: The methodology used to develop this tool takes into account expert opinions. Comprehensiveness of this tool makes it an ideal evaluation tool during self-evaluation or external quality assurance for longitudinal FDP. After its validation and testing, this practice guide can be used worldwide, along with the provided indicators which can be quantified and used to suit the local context.

Medical education in the United Arab Emirates: Challenges and opportunities.

Medical education (ME) in the United Arab Emirates (UAE) has a relatively short history that begins with the inception of the UAE almost 50 years ago. The UAE has made great strides in widening access to ME through the rapid implementation of national agendas aimed at advancing healthcare and expanding higher education, in addition to the presence of a strong infrastructure for privatization and business development. While progress is being made at all levels of ME, complex challenges for both undergraduate and postgraduate ME remain. Going forward, issues of standardization, quality, sustainability of academic and healthcare workforces, and research must continue to be addressed.

Virtual Clinical Encounter Examination (VICEE): A novel approach for assessing medical students' non-psychomotor clinical competency.

INTRODUCTION: The Corona Virus Disease-19 (COVID-19) pandemic disrupted medical education across the world. Online teaching has grown rapidly under lockdown. Yet the online approach for assessment presents a number of challenges, particularly when evaluating clinical competencies. The aim of this study was to investigate the feasibility, acceptability, reliability and validity of an online Virtual Clinical Encounter Examination (VICEE) to assess non-psychomotor competencies (non-procedure or manual skills) of medical students. METHOD: Sixty-one final year medical students took the VICEE as part of the final summative examination. A panel of faculty experts developed the exam cases and competencies. They administered the test online via real-time interaction with artificial intelligence (AI) based virtual patients, along with faculty and IT support. RESULTS: Student and faculty surveys demonstrated satisfaction with the experience. Confirmatory factor analysis supported convergent validity of VICEE with Direct Observation Clinical Encounter Examination (DOCEE), a previously validated clinical examination. The observed sensitivity was 81.8%, specificity 64.1% and likelihood ratio 12.6, supporting the ability of VICEE to diagnose 'clinical incompetence' among students. CONCLUSION: Our results suggest that online AI-based virtual patient high fidelity simulation may be used as an alternative tool to assess some aspects of non-psychometric competencies.

Measuring medical students' professional competencies in a problem-based curriculum: a reliability study.

BACKGROUND: Identification and assessment of professional competencies for medical students is challenging. We have recently developed an instrument for assessing the essential professional competencies for medical students in Problem-Based Learning (PBL) programs by PBL tutors. This study aims to evaluate the reliability and validity of professional competency scores of medical students using this instrument in PBL tutorials. METHODS: Each group of seven to eight students in PBL tutorials (Year 2, n = 46) were assessed independently by two faculty members. Each tutor assessed students in his/her group every five weeks on four occasions. The instrument consists of ten items, which measure three main competency domains: interpersonal, cognitive and professional behavior. Each item is scored using a five-point Likert scale (1 = poor, 5 = exceptional). Reliability of professional competencies scores was calculated using G-theory with raters nested in occasions. Furthermore, criterion-related validity was measured by testing the correlations with students' scores in written examination. RESULTS: The overall generalizability coefficient (G) of the professional competency scores was 0.80. Students' professional competencies scores (universe scores) accounted for 27% of the total variance across all score comparisons. The variance due to occasions accounted for 10%, while the student-occasion interaction was zero. The variance due to raters to occasions represented 8% of the total variance, and the remaining 55% of the variance was due to unexplained sources of error. The highest reliability measured was the interpersonal domain (G = 0.84) and the lowest reliability was the professional behavior domain (G = 0.76). Results from the decision (D) study suggested that an adequate dependability (G = 0.71) can be achieved by using one rater for five occasions. Furthermore, there was a positive correlation between the written examination scores and cognitive competencies scores (r = 0.46, P < 0.01), but not with the other two competency domains (interpersonal and professionalism). CONCLUSIONS: This study demonstrates that professional competency assessment scores of medical students in PBL tutorials have an acceptable reliability. Further studies for validating the instrument are required before using it for summative evaluation of students by PBL tutors.

Medical College of the Future: from Informative to Transformative.

The complexity of medical education and healthcare systems is a 'wicked problem'. Change will be a continuous iteration between evaluation and revaluation. Medicine is a social science. It is about people, societies and human interaction and communication. Medical College of the future should be developed in the light of social constructivism theories. Students from year one, day one, should be embedded in the work environment. Academic Healthcare Systems will be the norm not the exceptional. The training of students will be in all healthcare related facilities in the community. Public-private partnership in education and research will spread and become more regulated and encouraged. The students who want to be the future physicians should be selected differently. The curriculum will be more context related. Entrustable professional activities will be measured more frequently at different points of the students learning trajectory. Research and innovation will be integral to the students' learning experience. They should be exposed to how researchers think and behave and be embedded in a research environment. The medical college of the future will be using advanced technology which will be disruptive and transform existing educational models. E-Learning materials will be shared by consortia of collaborating medical colleges from all around the world. Artificial intelligence and machine learning will influence how students learn. National and international accreditation systems should ensure quality, but not stifle innovation. Physical learning spaces should reflect and express the underlying assumptions about "what is learning and teaching?". This should be reflected and expressed through the design of its buildings, infrastructure, technology, furniture and out of class learning environment. Predicting the future is difficult in a rapidly changing world. Next generation physicians should be competent in treating the next generation of patients while maintaining the health of the population.

Problem-based learning: where are we now? Guide supplement 36.3--Practical Application.

BACKGROUND: Problem-based learning (PBL) is a well used educational strategy, more frequently used in the pre-clinical phase rather than the clinical phase of medical student learning. This practical application explores its use in a Family Medicine undergraduate clerkship. METHOD: The PBL application is described during a 10- week rotation in Family Medicine for final year medical students. 217 students were asked to rank 24 statements describing their PBL experience on a five-point Likert scale. The grading was supplemented by one qualitative question. The clinical tutors were asked to write an evaluation statement on their experience. EVALUATION: The educational process was rated highly by students and faculty as a valuable learning methodology that enhanced their knowledge and understanding of patient problems, providing them with a more holistic approach to care. CONCLUSIONS: It would appear that PBL is infrequently used during clinical learning and specifically not in a Family Medicine clerkship. This small study supports its value to both student and faculty.

Adult learning theories: implications for learning and teaching in medical education: AMEE Guide No. 83.

There are many theories that explain how adults learn and each has its own merits. This Guide explains and explores the more commonly used ones and how they can be used to enhance student and faculty learning. The Guide presents a model that combines many of the theories into a flow diagram which can be followed by anyone planning learning. The schema can be used at curriculum planning level, or at the level of individual learning. At each stage of the model, the Guide identifies the responsibilities of both learner and educator. The role of the institution is to ensure that the time and resources are available to allow effective learning to happen. The Guide is designed for those new to education, in the hope that it can unravel the difficulties in understanding and applying the common learning theories, whilst also creating opportunities for debate as to the best way they should be used.

Assessment of clinical competencies using clinical images and videos "CIVA".

BACKGROUND: This paper describes an assessment approach of clinical competencies which widens the number of problems and tasks evaluated using videos and images. METHOD: Clinical Image and Video Assessment (CIVA) was used to assess clinical reasoning and decision making of final year medical students. Forty to fifty clinical videos and images supported by rich text vignette and reviewed by subject matter experts were selected based on examination blueprints for analysis. CIVA scores were correlated with OSCE, Direct Observation Clinical Encounter Exam (DOCEE) and written exam scores, using the 2-sided Pearson correlation analysis, and their reliability was analyzed using Cronbach's Alpha Coefficient. Furthermore, students personally evaluated the CIVA using a 5- point Likert scale. RESULTS: CIVA and OSCE scores showed a high correlation (r = 0.83) in contrast with the correlation scores of the written examination (r = .36) and the DOCEE (r = 0.35). Cronbach's Alpha for the OSCE and CIVA for the first batch was 0.71 and 0.78. As for the second batch it was 0.91 and 0.91 respectively. Eighty-two percent of students were very satisfied or satisfied with the CIVA process, contents and quality. CONCLUSIONS: A well constructed CIVA type assessment with a rich authentic vignette and good quality videos and images could be used to assess clinical reasoning and decision making of final year medical students. CIVA is an assessment tool which correlates well with OSCE, compliments the written and DOCEE and is easier to conduct at a possibly reduced cost.

Polarity Based Model for Guiding Medical School Strategy During Crisis - A Cross Sectional Qualitative Study.

BACKGROUND: Crises in academia can best be dealt with as a polarity that needs to be leveraged rather than a problem that needs to be solved. This work aimed at utilizing the Polarity Approach for Continuity and Transformation (PACT)™ to establish a guide for medical schools during times of crisis to minimize the effect of crisis-driven decisions on strategic growth. SUBJECTS AND METHODS: A qualitative study following the 5-Steps of the PACT process was conducted. A virtual mapping session was held with 108 medical educators from 22 countries to determine the upsides and downsides of strategic orientation and crisis management subsequently. RESULTS: Four polarity maps were generated identifying four tension areas; University reputation, mission, teams, and individuals followed by a 72-item assessment and another mapping session to map the warning signs and action steps. A comparison between private school scores and the whole cohort of respondents showed that private schools had the least problems in team-oriented work. CONCLUSION: This study highlighted the importance of taking measures to communicate the mission and supporting team functions inside universities either by enhancing resources or utilizing time and effort-saving strategies.

Is running a problem-based learning curriculum more expensive than a traditional Subject-Based Curriculum?

BACKGROUND: Running a problem-based learning (PBL) curriculum has been considered to be more expensive than a traditional Subject-Based Curriculum? This claim has not been well substantiated. AIM: To identify faculty educational activities (FEA) related to PBL, calculate its cost (faculty number and time) and compare it with a traditional Subject-Based Curriculum. METHODS: A structured questionnaire was developed through focused group discussion using a Delphi method to identify FEA and ensure face and content validity. For cost comparison, a hypothetic model was developed based on "what if" the curriculum was traditional subject-based, and running within the frame of the current regulations of University of Sharjah. RESULTS: A total of 23 faculty (16 full-time basic medical sciences and 7 clinical tutors) equal to 19.5 full-time equivalent (FTE) responded to the questionnaire. About five major domains and 26 sub-domains of FEA were identified. The calculated number of faculty needed for the hypothetical Subject-Based Curriculum should have been 19 FTEs. The average FEA hours per faculty per week per student in PBL was 17.41 and for the Subject-Based Curriculum would have been 17.85. CONCLUSION: PBL does not require an increase in number of faculty. The time spent by faculty on educational activities was similar in the two curriculum models. Although the cost of two strategies was similar, but the educational roles and the faculty engagement in education in a PBL curriculum is broader than in the traditional curriculum.