Influence of tutor interventions and group process on medical students' engagement in problem-based learning.

INTRODUCTION: Student engagement is influenced by several variables, among which are the teaching styles employed by faculty. In problem-based learning (PBL), the role of faculty is to facilitate the learning of the tutorial group as they work through clinical problems. However, the influence of tutor intervention styles and group process on engagement of students in PBL tutorials is unclear. METHODS: The study was conducted with year 2 and 3 medical students (n = 176) in PBL tutorial groups at the end of an integrated PBL course. Students evaluated their behavioural, cognitive and emotional engagement in PBL tutorials using a pre-validated 11-item questionnaire. Students also filled in a modified version of a previously published tutor intervention profile (TIP) questionnaire. The modified TIP questionnaire represents three constructs (1) steering the learning process (6 items), (2) stimulating student autonomy (4 items) and (3) establishing relatedness with students (3 items). In addition, PBL group process was evaluated using a 5-item nominal scale: (1) tutorial atmosphere, (2) listening and information sharing, (3) group performance, (4) decision making and (5) reaction to leadership. RESULTS: Establishing sense of relatedness in the group by PBL tutors was the most important predictor of emotional engagement (F = 41.213, ΔR2 = 0.191, ß = 0.438, P = 0.000). On the other hand, steering the learning process was a significant predictor of behavioural engagement (F = 19.0, ΔR2=0.098, ß = 0.314, P = 0.000). However, stimulating student autonomy was not a significant predictor of student engagement in PBL tutorials. On the other hand, enhancing the group process in PBL tutorials significantly predicts student engagement with strong impact on emotional and cognitive engagement of students. CONCLUSIONS: Establishing the sense of relatedness in the group and steering the learning process by PBL tutors as well as improving PBL group process are significant predictors of student engagement in PBL tutorials with emotional and cognitive engagement being the most sensitive variables affected.

Measuring sociocultural engagement of health professions education students: a psychometric study.

BACKGROUND: Sociocultural engagement of students refers to broadening viewpoints and providing awareness of, and respect for, diverse backgrounds and perspectives. However, there are no existing validated instruments in the literature for measuring sociocultural engagement of health professions education (HPE) students. Therefore, the aim of this study is to develop and validate a questionnaire designed to assess sociocultural engagement among HPE students. METHODS: The study included undergraduate HPE students (n = 683) at Gulf Medical University. The initial version of the sociocultural engagement of students' questionnaire (SESQ) was developed after extensive literature review and guided by the Global Learning Qualifications Framework. We then tested the content validity of the questionnaire by using focus group discussion with subject experts (n = 16) and pilot testing with students (n = 20). We distributed the content-validated version of the SESQ (16 items) to undergraduate students in six HPE colleges. To examine the construct validity and construct reliability of the questionnaire, we conducted exploratory factor analysis, followed by confirmatory factor analysis. RESULTS: Confirmatory factor analysis supported the two-factor structure which consists of 13 items with good fitness indices (χ2 = 214.35, df = 61, χ 2/df = 3.51, CFI = 0.98, RMSEA = 0.06, SRMR = 0.025, and AIC = 208.00). The two factors were sociocultural interactions (8 items) and sociocultural adaptation (5 items). The construct reliability of the total questionnaire is 0.97 and the two factors were 0.93 and 0.92 for sociocultural interactions and sociocultural adaptation, respectively. In addition, there were significant weak correlations between both factors of sociocultural engagement scores and student satisfaction with the university experience (r = .19 for each, P = .01). CONCLUSIONS: The sociocultural engagement of students' questionnaire exhibits good evidence of construct validity and reliability. Further studies will be required to test the validity of this questionnaire in other contexts.

Student engagement in health professions education: AMEE Guide No. 152.

This guide aims to support our colleagues to have comprehensive understanding of student engagement in health professions education. Despite the universal agreement about the significance of student engagement, there is lack of uniformity in conceptualizing and operationalizing this emerging construct. We review the theoretical basis explaining student engagement from three main perspectives: behavioral, psychological, and socio-cultural. In addition, we propose a contemporary and comprehensive framework for the student engagement in higher education, which is applicable to health professions education contexts. Drawing from this framework, we explain the conceptualization of the construct and its preceding factors, mediators, dimensions, spheres, and outcomes of student engagement. The proposed framework introduces student 'engagement through partnerships' as a novel component compared with the existing models of student engagement in higher education. This way, we are proposing a mixed model that not only considers the student as a 'customer' but also as a 'partner' in education. Engagement of students through partnerships include four areas: (1) provision of the education program, (2) scholarly research, (3) governance and quality assurance, and (4) community activities. This guide will provide practical applications on how to improve student engagement in health professions education. Finally, we highlight the current gaps in areas of research in the student engagement literature and suggested plans for future directions.[Box: see text].

Measurement of student engagement in health professions education: a review of literature.

Student engagement is a complex multidimensional construct that has attained great interest in health professions education (HPE). Definition and conceptualization of student engagement is an important step that should drive the development of the instruments for its measurement. We have recently proposed a comprehensive framework for student engagement in HPE with a definition of engagement as student investment of time and energy in academic and non-academic experiences that include learning, teaching, research, governance, and community activities. The dimensions of student engagement in this framework included the cognitive, affective, behavioral, agentic, and socio-cultural. Guided by the student engagement framework, this non-systematic review aims to identify, critically appraise, and summarize the existing methods for measuring student engagement in HPE. Extrapolating from higher education literature, we attempted to link the theoretical perspectives of student engagement with the published methods of its measurement in HPE context. In addition, we have described the different methods of measuring student engagement including self-report surveys, real time measures, direct observation, interviews/focus groups, and the use of multiple instruments. The span of engagement dimensions measured by self-report surveys ranges from one to five dimensions. However, measurement of agentic and sociocultural dimensions of engagement in HPE is still limited and further research is required. We have also reflected on the existing methods of measuring engagement of students as active partners in HPE. The review also describes the advantages, limitations, and psychometric properties of each method for measuring student engagement. We ended the review with a guiding conclusion on how to develop and select an instrument for measuring student engagement in HPE. Finally, we addressed the gaps in the literature about measuring engagement of HPE students and future research plans.

Construct validity of a questionnaire for measuring student engagement in problem-based learning tutorials.

BACKGROUND: Student engagement is student investment of time and energy in academic and non-academic experiences that include learning, teaching, research, governance, and community activities. Although previous studies provided some evidence of measuring student engagement in PBL tutorials, there are no existing quantitative studies in which cognitive, behavioral, and emotional engagement of students in PBL tutorials is measured. Therefore, this study aims to develop and examine the construct validity of a questionnaire for measuring cognitive, behavioral, and emotional engagement of students in PBL tutorials. METHODS: A 15-item questionnaire was developed guided by a previously published conceptual framework of student engagement. Focus group discussion (n = 12) with medical education experts was then conducted and the questionnaire was piloted with medical students. The questionnaire was then distributed to year 2 and 3 medical students (n = 176) in problem-based tutorial groups at the end of an integrated course, where PBL is the main strategy of learning. The validity of the internal structure of the questionnaire was tested by confirmatory factor analysis using structural equation modeling assuming five different models. Predictive validity evidence of the questionnaire was studied by examining the correlations between students' engagement and academic achievement. RESULTS: Confirmatory factor analysis indicates a good fit between the measurement and structural model of an 11-item questionnaire composed of a three-factor structure: behavioral engagement (3 items), emotional engagement (4 items), and cognitive engagement (4 items). Models in which the three latent factors were considered semi-independent provided the best fit. The construct reliabilities of behavioral, cognitive, and emotional factors were 0.82, 0.82, and 0.76, respectively. We failed however to find significant relationships between academic achievement and engagement. CONCLUSIONS: We found a strong evidence to support the construct validity of a three-factor structure of student engagement in PBL tutorial questionnaire. Further studies are required to test the validity of this instrument in other educational settings. The predictive validity is another area needing further scrutiny.

Student engagement in undergraduate medical education: A scoping review.

INTRODUCTION: Student engagement is defined as behavioural, cognitive and emotional aspects of students' academic experience in teaching, learning and research through interacting with other students, faculty and community. Despite the growing interest in the field of student engagement, medical education research in this area is still fragmented. This scoping review aims to contribute to the understanding of measurements, drivers and outcomes of medical student engagement. METHODS: The authors searched MEDLINE, PubMed, ProQuest, SCOPUS, ERIC, Science Direct and EBESCO for English articles published from 1990 until October 2021. In addition, we hand-searched key medical education journals and references in recently published articles. Using specific selection criteria, two authors independently reviewed the articles for eligibility, followed by data extraction using both quantitative and qualitative analysis. RESULTS: Of the 2136 retrieved articles, 51 studies were selected for the review, and 94% of them were published in the past 8 years. The prevailing measures of student engagement failed to cover the multidimensionality of the construct with more focus on the behavioural dimension. Quantitative methods represented two thirds of the studies with a higher frequency of cross-sectional designs and using self-reports. The identified drivers of medical students' engagement are directed mainly to modifying the context of the learning environment. These factors increased student engagement by fostering relevance of learning, positive student relationships with peers and faculty, and enhancing student agency, and sense of competence. Cognitive engagement appears to be a positive predictor of academic achievement, but the relationship with other dimensions of engagement is controversial. CONCLUSIONS: Student engagement in undergraduate medical education is an important, yet under-researched construct. The research that does exist suggests engagement to be malleable, something that can be modified by different types of interventions taking into consideration the context of education and practice. Further research is required, however, to address the gaps identified in this review.

The effect of access to electronic resources during examination on medical and dental students scores in summative assessment: quasi experimental study.

BACKGROUND: Access to electronic (E) resources has become an indispensable requirement in medical education and practice. OBJECTIVE: Our objective was to assess the effect of E-resources access during examination on end-course-exam scores of medical and dental students. METHODS: A quasi-experimental study which included two cohorts of medical (n = 106 & 85) and three cohorts of dental students (n = 66, 64 and 69) who took end-course- exams. Each exam was composed of two- parts (Part I and II), that encompassed equal number of questions and duration. Access to E-resources was allowed in part-II only. Items Difficulty Index (DI), Discrimination Index, (DisI), Point Biserial, (PBS) and cognitive level were determined. RESULTS: The study included 390 students. The proportion of items at various levels of DI, DisI, and PBS and the average values for item DI, DisI in both parts of each exam were comparable. The average scores in part-II were significantly higher than part-I (P < 0.001, < 0.001 and 0.04) and lower-order cognitive-level items scores were higher in three exams (P < 0.0001, 0.0001, 0.0001). Higher- order cognitive level items scores were comparable between part I and II in all courses. The significant factor for change in marks were questions cognitive level and type of the course. CONCLUSION: Access to E-resources during examination does not make a significant difference in scores of higher-order cognitive level items. Question cognitive level and course type were the significant factors for the change in exam scores when accessing E-resources. Time-restricted E-resources accessed tests that examine higher cognitive level item had no significant academic integrity drawback.

Implications of introducing case based radiological images in anatomy on teaching, learning and assessment of medical students: a mixed-methods study.

BACKGROUND: Introducing radiological anatomy in the preclinical curriculum can increase the understanding of Anatomy. Regardless of the integration when teaching anatomy, it is essential to maintain oversight as to what and how much is being taught. In addition, the knowledge requirements for preclinical students should be considered. The purpose of this kind of integration is that the student should be able to apply the knowledge which can help them better understand anatomy and not to make the course more challenging. This study aimed to understand whether adding radiological images would increase the difficulty level of the questions. METHODS: We introduced radiological images, including X Rays, CT scans and MRIs, when teaching anatomy in the preclinical curriculum. A class of 99 students were tested using A-type MCQs (n = 84). All 84 questions were categorized on whether they were case-based with or without a radiological image. The item analysis of both groups of test questions was then compared based on their difficulty and discrimination index. A qualitative student perception regarding the inclusion of radiological images in anatomy was also measured using a questionnaire with a 5-point Likert scale. RESULTS: The results showed that the performance level of the students was similar when comparing the test questions in both groups. The item analysis of the MCQs in the two groups revealed that by integrating radiological images when teaching anatomy, the various parameters in both groups of test questions were in the same range. More than 80% of the students felt that radiological images facilitate the achievement of learning outcomes and help to apply their knowledge in clinical contexts. The study's findings reported that the rate of satisfaction by including radiological images when teaching anatomy is high. CONCLUSION: Recognition and interpretation of images are essential in an undergraduate medical program. Students found it helpful when radiological images were introduced to them when teaching anatomy. Since the students' performance in summative exams in both groups of questions was in the same range, the findings also point out that adding radiological images when teaching anatomy does not increase the difficulty of the subject.

Measuring medical students' professional competencies in a problem-based curriculum: a reliability study.

BACKGROUND: Identification and assessment of professional competencies for medical students is challenging. We have recently developed an instrument for assessing the essential professional competencies for medical students in Problem-Based Learning (PBL) programs by PBL tutors. This study aims to evaluate the reliability and validity of professional competency scores of medical students using this instrument in PBL tutorials. METHODS: Each group of seven to eight students in PBL tutorials (Year 2, n = 46) were assessed independently by two faculty members. Each tutor assessed students in his/her group every five weeks on four occasions. The instrument consists of ten items, which measure three main competency domains: interpersonal, cognitive and professional behavior. Each item is scored using a five-point Likert scale (1 = poor, 5 = exceptional). Reliability of professional competencies scores was calculated using G-theory with raters nested in occasions. Furthermore, criterion-related validity was measured by testing the correlations with students' scores in written examination. RESULTS: The overall generalizability coefficient (G) of the professional competency scores was 0.80. Students' professional competencies scores (universe scores) accounted for 27% of the total variance across all score comparisons. The variance due to occasions accounted for 10%, while the student-occasion interaction was zero. The variance due to raters to occasions represented 8% of the total variance, and the remaining 55% of the variance was due to unexplained sources of error. The highest reliability measured was the interpersonal domain (G = 0.84) and the lowest reliability was the professional behavior domain (G = 0.76). Results from the decision (D) study suggested that an adequate dependability (G = 0.71) can be achieved by using one rater for five occasions. Furthermore, there was a positive correlation between the written examination scores and cognitive competencies scores (r = 0.46, P < 0.01), but not with the other two competency domains (interpersonal and professionalism). CONCLUSIONS: This study demonstrates that professional competency assessment scores of medical students in PBL tutorials have an acceptable reliability. Further studies for validating the instrument are required before using it for summative evaluation of students by PBL tutors.

Implementation and evaluation of a patient safety course in a problem-based learning program.

BACKGROUND: Since the development of the WHO patient safety curriculum guide, there has been insufficient reporting regarding the implementation and evaluation of patient safety courses in undergraduate problem-based learning (PBL) programs. This study is designed to implement a patient safety course to undergraduate students in a PBL medical school and evaluate this course by examining its effects on students' knowledge and satisfaction. METHODS: The target population included year 6 medical students (n = 71) at the Faculty of Medicine, Suez Canal University in Egypt. A 3-day course was conducted addressing three principal topics from the WHO patient safety curriculum guide. The methods of instruction included reflection on students' past experiences, PBL case discussions, and tasks with incident report cards. A pre- and post-test design was used to assess the effect of the course on students' knowledge of inpatient safety topics. Furthermore, students' perceptions of the quality of the course were assessed through a structured self-administered course evaluation questionnaire. RESULTS: The results of the pre- and post-test demonstrated a significant increase (P < 0.05) in the students' mean multiple choice question (MCQ) scores. The MCQ scores for "what is patient safety" topic increased by 50% (P < 0.01). Similarly, the MCQ scores for the "infection control" topic increased by 39% (P < 0.01), and scores for the "medication safety" topic increased by 45% (P < 0.01). The majority of students perceived the different aspects of the course positively, including the structure and introduction of the course (75%) and the communication skills (83.2%) and teamwork skills they had developed (94.4%). The findings of the incident report cards indicated that 46.7% of the students perceived that incidents most commonly take place in the emergency room while only 6.7% in the outpatient clinic. DISCUSSION: This patient safety education program within a PBL curriculum is positively perceived by students. Furthermore, patient safety education in clinical settings should focus on emergencies, where students perceive most errors.

Description of Medical Students' Behavioral, Cognitive, and Psychological Engagement with Faculty Online Teaching Styles.

Purpose: The teacher-student dyad is the heart of an institute. A teacher's instructional role significantly influences the student engagement that indirectly measures the institutional outcome. The online transition of medical education changed the milieu of medical education where a virtual link between teacher and student is the only hope for the learner to continue the learning. There were no studies on the relationship between student engagement and teaching styles during online medical education. Patients and Methods: The present study was an online survey-based descriptive study on medical undergraduates from two universities in the United Arab Emirates. Google forms containing consent statements, teaching styles inventory in higher education (TSIHE), and online student engagement questionnaires were emailed to all medical students requesting to participate in the study. Completed survey questionnaires were analyzed descriptively for the degree of student online engagement, and a chi-square test was used to correlate the relation between faculty online instructional methods and students' engagement. Results: A total of 423 of 927 students from two universities participated in the survey. There was no significant perception difference among the students from the two universities regarding their faculty online teaching styles. Thirty-three percent of students at first university and 41% of second university students showed engagement during online classes, which is statistically significant. However, the degree of students' disengagement from both universities was high compared to their engagement for online classes. Both university students' cognitive and behavioral engagement was moderate to strongly correlated with four domains of teaching style. Interestingly, there was no correlation between online faculty instructional methods and students' psychological engagement from both the universities. Conclusion: The present study establishes the relationship between students' cognitive and behavioral engagement and teaching instructional practices. However, there is a need to develop robust evidence on students' psychological engagement and the influencing factors during online and blended contexts.

Correction: Using Assessment Design Decision Framework in understanding the impact of rapid transition to remote education on student assessment in health-related colleges: A qualitative study.

[This corrects the article DOI: 10.1371/journal.pone.0254444.].

Faculty and Students' Perceptions About Online Teaching Styles of Faculty in Large Group Lectures.

Introduction: An unprecedented disruption in medical education worldwide was caused due to the Covid 19 pandemic. Online teaching has become one of the primary forms of education. In this paper, we aimed to understand the faculty and students' perceptions about the teaching styles adopted by the faculty during online large group teaching from two universities in the United Arab Emirates. Patients and Methods: This is a cross-sectional opinion survey-based study. A pre-validated teaching style inventory in higher education (TSIHE) questionnaire was emailed to the students and faculty involved in large group online learning and teaching activities during the lock-down period of the COVID 19 pandemic from two different medical Universities in the United Arab Emirates. A total of 423 students and 57 faculty participated in the study. Descriptive and Inferential statistics were used to analyze the data. Results: There was a significant difference between the faculty and students on the perception of faculty online teaching styles. The difference was significant in teacher-student interaction and teacher structuring styles. In contrast, faculty and students' perceptions were comparable for decision-making negotiation and behavioral control teaching styles. Conclusion: The study found that there was a significant perception differences between the faculty and students on the faculty teaching style during online large group teaching. Importantly, faculty perceived that they were more emotionally attached and more flexible in teacher-student interaction and teaching structuring during online teaching than the student perceived.

Psychometric Properties of Visual Indicators of Teaching and Learning Success "VITALS" Instrument for Evaluation of Clinical Teachers.

PURPOSE: We have previously developed an instrument for students' evaluation of clinical teachers that we called Visual Indicators of Clinical Teaching and Learning Success (VITALS). This study measures the reliability of VITALS as an instrument for student evaluation of clinical tutors. Additionally, the study explores the minimum number of student raters necessary for an acceptable reliability, and provides evidence of construct validity of the evaluation scores. MATERIALS AND METHODS: This retrospective study included 1825 evaluation forms completed by medical students evaluating clinical tutors using the VITALS instrument. Reliability was measured by applying generalizability theory (G-theory) analysis using a two-facet design (raters and items). A D-study was used to determine the minimum number of raters required to achieve a reliability ≥0.80. Face validity was tested by measuring tutors' degree of agreement with the items of the study instrument. RESULTS: The overall G-coefficient was 0.89. The subject of measurement (clinical tutors' scores) represented 15.8% of the variance across all tutors and items. The variance due to the interaction between raters (students) and tutors contributed to 43.5%, while the variance due to items was negligible. The remaining 40% of the variance was due to unexplained sources of error. The D-study demonstrated that a minimum of 12 raters (students) are required to achieve a reliability of 0.80. Finally, most of the clinical tutors agreed that all items in the instrument were appropriate. CONCLUSION: We demonstrate that VITALS exhibits good psychometric properties. There should be at least 12 students rating each clinical tutor to have an acceptable level of reliability for the study instrument. Face validity of the study instrument was evidenced by its high level of approval among clinical tutors.

Using Assessment Design Decision Framework in understanding the impact of rapid transition to remote education on student assessment in health-related colleges: A qualitative study.

Maintaining integrity and validity with online assessment is a significant issue that is well documented. Overt policies encouraging educators to adopt e-Learning and implement digital services coupled with the dramatic change in the education system in response to the challenges posed by COVID-19, has furthered the demand for evidence-based approaches for the planning and delivery of assessments. This study employed the Assessment Design Decision Framework (ADDF), a theoretical model that considers key aspects of assessment design, to retrospectively investigate from a multi-stakeholder perspective the assessments implemented following the rapid transition to remote learning during the COVID-19 pandemic. One-to-one semi-structured interviews were conducted with faculty and students from the Colleges of Pharmacy, Medicine and Health Sciences. After inductive and deductive thematic analysis three major themes were identified. These reflected on the impact of sudden transition on assessment design and assessment plan; changing assessment environment; and faculty-student assessment related interactions which included feedback. The use of a comprehensive validated framework such as ADDF, to plan assessments can improve validity and credibility of assessments. The strengths of this study lie in the innovative adoption of the ADDF to evaluate assessment design decisions from both an educator and student perspective. Further, the data yielded from this study offers novel validation of the use of ADDF in circumstances necessitating rapid transition, and additionally identifies a need for greater emphasis to be attributed to the significance of timeliness of the various activities that are advocated within the framework.

Concept mapping assessment in a problem-based medical curriculum.

BACKGROUND: In the problem-based learning (PBL) medical curriculum at the Arabian Gulf University in Bahrain, students construct concept maps related to each case they study in PBL tutorials. AIM: To evaluate the interrater reliability and predictive validity of concept map scores using a structured assessment tool. METHODS: We examined concept maps of the same cohort of students at the beginning (year 2) and end (year 4) of the pre-clerkship phase, where PBL is the main method of instruction. Concept maps were independently evaluated by five raters based on valid selection of concepts, hierarchical arrangement of concepts, integration, relationship to the context of the problem, and degree of student creativity. A 5-point Likert scale was used to evaluate each criterion. Interrater reliability of the instrument was determined using the intraclass correlation coefficient (ICC) and predictive validity was measured by testing the correlations of concept map scores with summative examination scores. RESULTS: The ICC of the concept map scores in year 2 was 0.75 (95% CI, 0.67-0.81) and in year 4 was 0.69 (95% CI, 0.59-0.77). Overall concept maps scores of year 4 students were significantly higher compared with year 2 students (p < 0.001, effect size = 0.5). The relationship between the students' scores in concept maps and their scores in summative examination varied from no to mild correlation. CONCLUSION: The interrater reliability of concept map scores in this study is good to excellent. However, further studies are required to test the generalizability and validity of assessment using this tool.

Construct Validity of an Instrument for Assessment of Reflective Writing-Based Portfolios of Medical Students.

PURPOSE: Assessment of reflective writing for medical students is challenging, and there is lack of an available instrument with good psychometric properties. The authors developed a new instrument for assessment of reflective writing-based portfolios and examined the construct validity of this instrument. METHODS: After an extensive literature review and pilot testing of the instrument, two raters assessed the reflective writing-based portfolios from years 2 and 3 medical students (n=135) on three occasions. The instrument consists of three criteria: organization, description of an experience and reflection on the experience. We calculated the reliability of scores using generalizability theory with a fully crossed design and two facets (raters and occasions). In addition, we measured criterion validity by testing correlations with students' scores using other assessment methods. RESULTS: The dependability (Φ) coefficient of the portfolio scores was 0.75 using two raters on three occasions. Students' portfolio scores represented 46.6% of the total variance across all score comparisons. The variance due to occasions was negligible, while the student-occasion interaction was small. The variance due to student-rater interaction represented 17.7%, and the remaining 27.7% of the variance was due to unexplained sources of error. The decision (D) study suggested that an acceptable dependability (Φ = 0.70 and 0.72) can be achieved by using two raters for one and two occasions, respectively. Finally, we found moderate to large effect-size correlations between students' scores in reflective writing-based portfolios and communication skills (r = 0.47) and PBL tutorials (r = 0.50). CONCLUSION: We demonstrated the presence of different sources of evidence that support construct validity of the study instrument. Further studies are warranted before utilizing this instrument for summative assessment of students' reflective writing-based portfolios in other medical schools.

Lecturing skills as predictors of tutoring skills in a problem-based medical curriculum.

PURPOSE: Recruitment of tutors to work in problem-based learning (PBL) programs is challenging, especially in that most of them are graduated from discipline-based programs. Therefore, this study aims at examining whether lecturing skills of faculty could predict their PBL tutoring skills. METHODS: This study included evaluation of faculty (n=69) who participated in both tutoring and lecturing within particular PBL units at the College of Medicine and Medical Sciences (CMMS), Arabian Gulf University, Bahrain. Each faculty was evaluated by medical students (n=45±8 for lecturing and 8±2 for PBL tutoring) using structured evaluation forms based on a Likert-type scale (poor to excellent). The prediction of tutoring skills using lecturing skills was statistically analyzed using stepwise linear regression. RESULTS: Among the parameters used to judge lecturing skills, the most important predictor for tutoring skills was subject matter mastery in the lecture by explaining difficult concepts and responding effectively to students' questions. Subject matter mastery in the lecture positively predicted five tutoring skills and accounted for 25% of the variance in overall effectiveness of the PBL tutors (F=22.39, P=0.000). Other important predictors for tutoring skills were providing a relaxed class atmosphere and effective use of audiovisual aids in the lecture. CONCLUSION: Predicting the tutoring skills based on lecturing skills could have implications for recruiting tutors in PBL medical programs and for tutor training initiatives.

Do medical students' scores using different assessment instruments predict their scores in clinical reasoning using a computer-based simulation?

PURPOSE: The development of clinical problem-solving skills evolves over time and requires structured training and background knowledge. Computer-based case simulations (CCS) have been used for teaching and assessment of clinical reasoning skills. However, previous studies examining the psychometric properties of CCS as an assessment tool have been controversial. Furthermore, studies reporting the integration of CCS into problem-based medical curricula have been limited. METHODS: This study examined the psychometric properties of using CCS software (DxR Clinician) for assessment of medical students (n=130) studying in a problem-based, integrated multisystem module (Unit IX) during the academic year 2011-2012. Internal consistency reliability of CCS scores was calculated using Cronbach's alpha statistics. The relationships between students' scores in CCS components (clinical reasoning, diagnostic performance, and patient management) and their scores in other examination tools at the end of the unit including multiple-choice questions, short-answer questions, objective structured clinical examination (OSCE), and real patient encounters were analyzed using stepwise hierarchical linear regression. RESULTS: Internal consistency reliability of CCS scores was high (α=0.862). Inter-item correlations between students' scores in different CCS components and their scores in CCS and other test items were statistically significant. Regression analysis indicated that OSCE scores predicted 32.7% and 35.1% of the variance in clinical reasoning and patient management scores, respectively (P<0.01). Multiple-choice question scores, however, predicted only 15.4% of the variance in diagnostic performance scores (P<0.01), while students' scores in real patient encounters did not predict any of the CCS scores. CONCLUSION: Students' scores in OSCE are the most important predictors of their scores in clinical reasoning and patient management using CCS. However, real patient encounter assessment does not appear to test a construct similar to what is tested in CCS.