Global Perceptions on Social Accountability and Outcomes: A Survey of Medical Schools.

Phenomenon: Social accountability has become a universal component in medical education. However, medical schools have little guidance for operationalizing and applying this concept in practice. This study explored institutional practices and administrative perceptions of social accountability in medical education. Approach: An online survey was distributed to a purposeful sample of English-speaking undergraduate medical school deans and program directors/leads from 245 institutions in 14 countries. The survey comprised of 38-items related to program mission statements, admission processes, curricular content, and educational outcomes. Survey items were developed using previous literature and categorized using a context-input-process-products (CIPP) evaluation model. Exploratory Factor Analysis (EFA) was used to assess the inter-relationship among survey items. Reliability and internal consistency of items were evaluated using McDonald's Omega. Findings: Results from 81 medical schools in 14 countries collected between February and June 2020 are presented. Institutional commonalities of social accountability were observed. However, our findings suggest programs focus predominately on educational inputs and processes, and not necessarily on outcomes. Findings from our EFA demonstrated excellent internal consistency and reliability. Four-factors were extracted: (1) selection and recruitment; (2) institutional mandates; (3) institutional activities; and (4) community awareness, accounting for 71% of the variance. McDonald's Omega reliability estimates for subscales ranged from 0.80-0.87. Insights: This study identified common practices of social accountability. While many medical schools expressed an institutional commitment to social accountability, their effects on the community remain unknown and not evaluated. Overall, this paper offers programs and educators a psychometrically supported tool to aid in the operationalization and reliability of evaluating social accountability.

Limited effects from professional identity formation-oriented intervention on self-regulated learning in a preclinical setting: a randomized-controlled study in Japan.

BACKGROUND: Developing self-regulated learning in preclinical settings is important for future lifelong learning. Previous studies indicate professional identity formation, i.e., formation of self-identity with internalized values and norms of professionalism, might promote self-regulated learning. We designed a professional identity formation-oriented reflection and learning plan format, then tested effectiveness on raising self-regulated learning in a preclinical year curriculum. METHODS: A randomized controlled crossover trial was conducted using 112 students at Jichi Medical University. In six one-day problem-based learning sessions in a 7-month pre-clinical year curriculum, Groups A (n = 56, female 18, mean age 21.5y ± 0.7) and B (n = 56, female 11, mean age 21.7y ± 1.0) experienced professional identity formation-oriented format: Group A had three sessions with the intervention format in the first half, B in the second half. Between-group identity stages and self-regulated learning levels were compared using professional identity essays and the Motivated Strategies for Learning Questionnaire. RESULTS: Two-level regression analyses showed no improvement in questionnaire categories but moderate improvement of professional identity stages over time (R2 = 0.069), regardless of timing of intervention. CONCLUSIONS: Professional identity moderately forms during the pre-clinical year curriculum. However, neither identity nor self-regulated learning is raised significantly by limited intervention.

Simulation-based education for novices: complex learning tasks promote reflective practice.

CONTEXT: Simulated clinical immersion (SCI), in which clinical situations are simulated in a realistic environment, safely and gradually exposes novices to complex problems. Given their limited experience, undergraduate students can potentially be quite overwhelmed by SCI learning tasks, which may result in misleading learning outcomes. Although task complexity should be adapted to the learner's level of expertise, many factors, both intrinsic and extraneous to the learning task, can influence perceived task complexity and its impact on cognitive processes. OBJECTIVES: The purpose of this mixed-methods study was to understand the effects of task complexity on undergraduate pharmacy students' cognitive load, task performance and perception of learning in SCI. METHODS: A total of 167 second-year pharmacy students were randomly assigned to undertake one simple and one complex learning task in SCI consecutively. Participants' cognitive load was measured after each task and debriefing. Task performance and time on task were also assessed. As part of a sequential explanatory design, semi-structured interviews were conducted with students showing maximal variations in intrinsic cognitive load to elucidate their perceptions of learning when dealing with complexity. RESULTS: Although the complex task generated significantly higher cognitive load and time on task than the simple task, performance was high for both tasks. Qualitative results revealed that a lack of clinical experience, an unfamiliar resource in the environment and the constraints inherent to SCI, such as time limitations, hindered the clinical reasoning process and led to poorer self-evaluation of performance. Simple tasks helped students gain more self-confidence, whereas complex tasks further encouraged reflective practice during debriefings. CONCLUSIONS: Although complex tasks in SCI were more cognitively demanding and took longer to execute, students indicated that they learned more from them than they did from simple tasks. Complex tasks constitute an additional challenge in terms of clinical reasoning and thus provide a more valuable learning experience from the student's perspective.

Does changing from a teacher-centered to a learner-centered context promote self-regulated learning: a qualitative study in a Japanese undergraduate setting.

BACKGROUND: Previous studies indicate that a teacher-centered context could hinder undergraduates from self-regulated learning (SRL), whereas a learner-centered context could promote SRL. However, SRL development between a teacher-centered and a learner-centered context has not directly compared in undergraduate settings. Also, it is still unclear how a contextual change toward learner-centered learning could influence SRL in students, who are strongly accustomed to teacher-centered learning. METHODS: We conducted three focus groups that examined 13 Japanese medical students who left a traditional curriculum composed of didactic lectures and frequent summative tests and entered a seven-month elective course (Free Course Student Doctor or FCSD). The FCSD emphasizes student-designed individualized learning with support and formative feedback from mentors chosen by students' preference. We also conducted two focus groups that examined 7 students who remained in the teacher-centered curriculum during the same period. Students were asked to discuss their 1) motivation, 2) learning strategies, and 3) self-reflection on self-study before and during the period. Data were analyzed using thematic analysis and code comparison between the two cohorts. RESULTS: The non-FCSD participants described their motivational status as being one among a crowd set by the teacher's yardstick. Their reflection focused on minimizing the gap between themselves and the teacher-set yardstick with strategies considered monotonous and homogeneous (e.g. memorization). FCSD participants described losing the teacher-set yardstick and constructing their future self-image as an alternative yardstick. They compared gaps between their present status and future self-image by self-reflection. To fill these gaps, they actively employed learning strategies used by doctors or mentors, leading to diversification of their learning strategies. CONCLUSIONS: A contextual change toward learner-centered learning could promote SRL even in students strongly accustomed to teacher-centered learning. In the learner-centered context, students began to construct their self-image, conduct self-reflection, and seek diverse learning strategies by referring to future 'self' models.

Contextual attributes promote or hinder self-regulated learning: A qualitative study contrasting rural physicians with undergraduate learners in Japan.

BACKGROUND AND OBJECTIVES: Previous studies support the notion that East Asian medical students do not possess sufficient self-regulation for postgraduate clinical training. However, some East Asian physicians who are employed in geographically isolated and educationally underserved rural settings can self-regulate their study during the early phase of their postgraduate career. To explore the contextual attributes that contribute to self-regulated learning (SRL), we examined the differences in self-regulation between learning as an undergraduate and in a rural context in East Asia. METHODS: We conducted interviews and diary data collection among rural physicians (n = 10) and undergraduates (n = 11) in Japan who undertook self-study of unfamiliar diseases. We analyzed three domains of Zimmerman's definition of SRL: learning behaviors, motivation, and metacognition using constructivist grounded theory. RESULTS: Rural physicians recognized their identity as unique, and as professionals with a central role of handling diseases in the local community by conducting self-study. They simultaneously found themselves being at risk of providing inappropriate aid if their self-study was insufficient. They developed strategic learning strategies to cope with this high-stakes task. Undergraduates had a fear of being left behind and preferred to remain as one of the crowd with students in the same school year. Accordingly, they copied the methods of other students for self-study and used monotonous and homogeneous strategies. CONCLUSIONS: Different learning contexts do not keep East Asian learners from being self-regulated. Awareness of their unique identity leads them to view learning tasks as high-stakes, and to initiate learning strategies in a self-regulated manner. Teacher-centered education systems cause students to identify themselves as one of the crowd, and tasks as low-stakes, and to accordingly employ non-self-regulated strategies.

Ward round simulation in final year medical students: Does it promote students learning?

INTRODUCTION: Ward round skills are essential for doctors in hospital settings. Literature shows medical students' deficiencies in these skills. Simulation has been used to train these skills. However, exposing learners to simulation at an early stage may be associated with a high cognitive load and limited learning. This study aims to determine how students experience this load and its interplay with performance and which factors promote and impair learning. METHODS: Fifty-six final year medical students participated in a simulated ward round training exercise. Both students' performance and cognitive load were measured to determine if there was any correlation and interviews were carried out to understand which factors support and impair learning. RESULTS: Performance scores revealed deficiencies in ward round skills. Students experienced a cognitive load that weakly correlated with performance. Qualitative findings provided important insights into simulated ward-based learning. It is clear that well-designed clinical scenarios, prioritization tasks, teamwork and feedback support students' learning process whereas distractions impair learning. CONCLUSIONS: WRS proved to be a good teaching method to improve clinical skills at this stage as the cognitive load is not too high to impair learning. Hence, including tasks in the simulation design can enhance the learning process.

Why are children overconfident? Developmental differences in the implementation of accessibility cues when judging concept learning.

Children are often overconfident when monitoring their learning, which is harmful for effective control and learning. The current study investigated children's (N=167, age range 7-12years) judgments of learning (JOLs) when studying difficult concepts. The main aims were (a) to investigate how JOL accuracy is affected by accessibility cues and (b) to investigate developmental changes in implementing accessibility cues in JOLs. After studying different concepts, children were asked to generate novel sentences and then to make JOLs, select concepts for restudy, and take a final test. Overconfidence for incorrect and incomplete test responses was reduced for older children in comparison with younger children. For older age groups, generating a sentence led to greater overconfidence compared with not being able to generate a sentence, which indicates that older children relied more on accessibility cues when making JOLs. This pattern differed in the youngest age group; younger children were generally overconfident regardless of whether they had generated sentences or not. Overconfidence was disadvantageous for effective control of learning for all age groups. These findings imply that instructions to encourage children to avoid metacognitive illusions need to be adapted to children's developmental stage.

Task Demands in OSCEs Influence Learning Strategies.

THEORY: Models on pre-assessment learning effects confirmed that task demands stand out among the factors assessors can modify in an assessment to influence learning. However, little is known about which tasks in objective structured clinical examinations (OSCEs) improve students' cognitive and metacognitive processes. Research is needed to support OSCE designs that benefit students' metacognitive strategies when they are studying, reinforcing a hypothesis-driven approach. With that intent, hypothesis-driven physical examination (HDPE) assessments ask students to elicit and interpret findings of the physical exam to reach a diagnosis ("Examine this patient with a painful shoulder to reach a diagnosis"). HYPOTHESES: When studying for HDPE, students will dedicate more time to hypothesis-driven discussions and practice than when studying for a part-task OSCE ("Perform the shoulder exam"). It is expected that the whole-task nature of HDPE will lead to a hypothesis-oriented use of the learning resources, a frequent use of adjustment strategies, and persistence with learning. METHOD: In a mixed-methods study, 40 medical students were randomly paired and filmed while studying together for two hypothetical OSCE stations. Each 25-min study period began with video cues asking to study for either a part-task OSCE or an HDPE. In a crossover design, sequences were randomized for OSCEs and contents (shoulder or spine). Time-on-task for discussions or practice were categorized as "hypothesis-driven" or "sequence of signs and maneuvers." Content analysis of focus group interviews summarized students' perception of learning resources, adjustment strategies, and persistence with learning. RESULTS: When studying for HDPE, students allocate significantly more time for hypothesis-driven discussions and practice. Students use resources contrasting diagnoses and report persistence with learning. When studying for part-task OSCEs, time-on-task is reversed, spent on rehearsing a sequence of signs and maneuvers. CONCLUSIONS: OSCEs with similar contents but different task demands lead to opposite learning strategies regarding how students manage their study time. Measuring pre-assessment effects from a metacognitive perspective provides empirical evidence to redesign assessments for learning.

The simulated clinical environment: Cognitive and emotional impact among undergraduates.

CONTEXT: Simulated clinical immersion (SCI) is used in undergraduate healthcare programs to expose the learner to real-life situations in authentic simulated clinical environments. For novices, the environment in which the simulation occurs can be distracting and stressful, hence potentially compromising learning. OBJECTIVES: This study aims to determine whether SCI (with environment) imposes greater extraneous cognitive load and stress on undergraduate pharmacy students than simulated patients (SP) (without environment). It also aims to explore how features of the simulated environment influence students' perception of learning. METHODS: In this mixed-methods study, 143 undergraduate pharmacy students experienced both SCI and SP in a crossover design. After the simulations, participants rated their cognitive load and emotions. Thirty-five students met in focus groups to explore their perception of learning in simulation. RESULTS: Intrinsic and extraneous cognitive load and stress scores in SCI were significantly but modestly higher compared to SP. Qualitative findings reveal that the physical environment in SCI generated more stress and affected students? focus. In SP, students concentrated on clinical reasoning. SCI stimulated a focus on data collection but impeded in-depth problem solving processes. CONCLUSION: The physical environment in simulation influences what and how students learn. SCI was reported as more cognitively demanding than SP. Our findings emphasize the need for the development of adapted instructional design guidelines in simulation for novices.

Twelve tips for medical curriculum design from a cognitive load theory perspective.

During their course, medical students have to become proficient in a variety of competencies. For each of these competencies, educational design can use cognitive load theory to consider three dimensions: task fidelity: from literature (lowest) through simulated patients (medium) to real patients (highest); task complexity: the number of information elements in a learning task; and instructional support: from worked examples (highest) through completion tasks (medium) to autonomous task performance (lowest). One should integrate any competency into a medical curriculum such that training in that competency facilitates the students' journey that starts from high instructional support on low-complexity low-fidelity learning tasks all the way to high-complexity tasks in high-fidelity environments carried out autonomously. This article presents twelve tips on using cognitive load theory or, more specifically, a set of four tips for each of task fidelity, task complexity, and instructional support, to achieve that aim.

Influences of OSCE design on students' diagnostic reasoning.

CONTEXT: Some characteristics of assessments exert a strong influence on how students study. Understanding these pre-assessment learning effects is of key importance to the designing of medical assessments that foster students' reasoning abilities. Perceptions of the task demands of an assessment significantly influence students' cognitive processes. However, why and how certain tasks positively 'drive' learning remain unknown. Medical tasks can be assessed as coherent meaningful whole tasks (e.g. examining a patient based on his complaint to find the diagnosis) or can be divided into simpler part tasks (e.g. demonstrating the physical examination of a pre-specified disease). Comparing the benefits of whole-task and part-task assessments in a randomised controlled experiment could guide the design of 'assessments for learning'. OBJECTIVES: The purpose of this study was to determine whether the knowledge that an objective structured clinical examination (OSCE) will contain whole tasks, as opposed to part tasks, increases the use of diagnostic reasoning by medical students when they study for this assessment. METHODS: In this randomised, controlled, mixed-methods experiment, 40 medical students were randomly paired and filmed while studying together for two imminent physical examination OSCE stations. Each 25-minute study period began with video cues and ended with a questionnaire on cognitive loads. Cues disclosed either a part-task OSCE station (examination of a healthy patient) or a whole-task OSCE station (hypothesis-driven physical examination [HDPE]). In a crossover design, sequences were randomised for both task and content (shoulder or spine). Two blinded and independent authors scored all 40 videos in distinct randomised orders, listening to participants studying freely. Mentioning a diagnosis in association with a sign was scored as a backward association, and the opposite was scored as a forward association; both revealed the use of diagnostic reasoning. Qualitative data were obtained through group interviews. RESULTS: Studying for whole-task OSCE stations resulted in a greater use of diagnostic reasoning. Qualitative data triangulate these findings and show the precedence of cues sourced from the 'student grapevine'. CONCLUSIONS: In comparison with 'traditional' part-task OSCEs, whole-task OSCEs like the HDPE increase students' use of diagnostic reasoning during study time.

Effects of learning content in context on knowledge acquisition and recall: a pretest-posttest control group design.

BACKGROUND: It is generally assumed that learning in context increases performance. This study investigates the relationship between the characteristics of a paper-patient context (relevance and familiarity), the mechanisms through which the cognitive dimension of context could improve learning (activation of prior knowledge, elaboration and increasing retrieval cues), and test performance. METHODS: A total of 145 medical students completed a pretest of 40 questions, of which half were with a patient vignette. One week later, they studied musculoskeletal anatomy in the dissection room without a paper-patient context (control group) or with (ir)relevant-(un)familiar context (experimental groups), and completed a cognitive load scale. Following a short delay, the students completed a posttest. RESULTS: Surprisingly, our results show that students who studied in context did not perform better than students who studied without context. This finding may be explained by an interaction of the participants' expertise level, the nature of anatomical knowledge and students' approaches to learning. A relevant-familiar context only reduced the negative effect of learning the content in context. Our results suggest discouraging the introduction of an uncommon disease to illustrate a basic science concept. Higher self-perceived learning scores predict higher performance. Interestingly, students performed significantly better on the questions with context in both tests, possibly due to a 'framing effect'. CONCLUSIONS: Since studies focusing on the physical and affective dimensions of context have also failed to find a positive influence of learning in a clinically relevant context, further research seems necessary to refine our theories around the role of context in learning.

Factors Influencing Seminar Learning and Academic Achievement.

Many veterinary curricula use seminars, interactive educational group formats in which some 25 students discuss questions and issues relating to course themes. To get indications on how to optimize the seminar learning process for students, we aimed to investigate relationships between factors that seem to be important for the seminar learning process, and to determine how these seminar factors account for differences in students' achievement scores. A 57-item seminar evaluation (USEME) questionnaire was administered to students right after they attended a seminar. In total, 80 seminars distributed over years 1, 2, and 3 of an undergraduate veterinary medicine curriculum were sampled and 988 questionnaires were handed in. Principal factor analysis (PFA) was conducted on 410 questionnaires to examine which items could be grouped together as indicators of the same factor, and to determine correlations between the derived factors. Multilevel regression analysis was performed to explore the effects of these seminar factors and students' prior achievement scores on students' achievement scores. Within the questionnaire, four factors were identified that influence the seminar learning process: teacher performance, seminar content, student preparation, and opportunities for interaction within seminars. Strong correlations were found between teacher performance, seminar content, and group interaction. Prior achievement scores and, to a much lesser extent, the seminar factor group interaction appeared to account for differences in students' achievement scores. The factors resulting from the present study and their relation to the method of assessment should be examined further, for example, in an experimental setup.

Outcome and impact of Master of Public Health programs across six countries: education for change.

BACKGROUND: The human resources for health crisis has highlighted the need for high-level public health education to add specific capacities to the workforce. Recently, it was questioned whether Master of Public Health (MPH) training prepared graduates with competencies relevant to low- and middle-income countries (LMICs). This study aims to examine the influence of the MPH programs geared towards LMICs offered in Vietnam, China, South Africa, Mexico, Sudan, and the Netherlands on graduates' careers, application of acquired competencies, performance at the workplace, and their professional contribution to society. METHODS: A self-administered questionnaire was sent to graduates from six MPH programs. Frequency distributions of the answers were calculated, and a bivariate analysis and logistic regression of certain variables was performed. RESULTS: The response rate was 37.5%. Graduates reported change in leadership (69%), in technical position (69%), acquiring new responsibilities (80%), and increased remuneration (63%); they asserted that MPH programs contributed significantly to this. Graduates' attribution of their application of 7 key competencies 'substantially to the MPH program' ranged from 33% to 48%. Of the 26 impact variables, graduates attributed the effect they had on their workplace substantially to the MPH program; the highest rated variable ranged from 31% to 73% and the lowest ranged from 9% to 43%. Of the 10 impact variables on society, graduates attributed the effect they had on society substantially to the MPH program; for the highest rated variable (13% to 71%); for the lowest rated variable (4% to 42%). Candidates' attribution of their application of acquired competencies as well as their impact at the workplace varied significantly according to institution of study and educational background. CONCLUSIONS: This study concludes that these MPH programs contribute to improving graduates' careers and to building leadership in public health. The MPH programs contribute to graduates' application of competencies. MPH programs contribute substantially towards impact variables on the workplace, such as development of research proposals and reporting on population health needs, and less substantially to their impact on society, such as contributing equitable access to quality services. Differences reported between MPH programs merit further study. The results can be used for curriculum reform.

Midterm peer feedback in problem-based learning groups: the effect on individual contributions and achievement.

Even though peer process feedback is an often used tool to enhance the effectiveness of collaborative learning environments like PBL, the conditions under which it is best facilitated still need to be investigated. Therefore, this study investigated the effects of individual versus shared reflection and goal setting on students' individual contributions to the group and their academic achievement. In addition, the influence of prior knowledge on the effectiveness of peer feedback was studied. In this pretest-intervention-posttest study 242 first year students were divided into three conditions: condition 1 (individual reflection and goal setting), condition 2 (individual and shared reflection and goal setting), and condition 3 (control group). Results indicated that the quality of individual contributions to the tutorial group did not improve after receiving the peer feedback, nor did it differ between the three conditions. With regard to academic achievement, only males in conditions 1 and 2 showed better academic achievement compared with condition 3. However, there was no difference between both ways of reflection and goal setting with regard to achievement, indicating that both ways are equally effective. Nevertheless, it is still too early to conclude that peer feedback combined with reflection and goal setting is not effective in enhancing students' individual contributions. Students only had a limited number of opportunities to improve their contributions. Therefore, future research should investigate whether an increase in number of tutorial group meetings can enhance the effectiveness of peer feedback. In addition, the effect of quality of reflection and goal setting could be taken into consideration in future research.

Investigating teaching performance in seminars; a questionnaire study with a multi-level approach.

BACKGROUND: Teachers play an important role in seminars as facilitators and content experts. However, contextual factors like students' preparation, group size, group interaction, and content appear to influence their performance. Understanding the impact of these contextual factors on students' perception of teaching performance may help to further understand seminar teaching. Besides that, it may help curriculum organisers and teachers to get more insight in how to optimise their versatile role in seminars. The aim of this study is to investigate how students' perception of teaching performance in seminars is explained by students' extent of preparation, seminar group size, group interaction, and content. METHODS: The Utrecht Seminar Evaluation (USEME) questionnaire was used to collect information on teaching performance and the aforementioned explanatory variables. To account for intra-student, intra-seminar, and intra-teacher correlation in the data, multilevel regression was used to analyse 988 completed questionnaires in 80 seminars with 36 different teachers. RESULTS: Group interaction and seminar content had large (B = 0.418) and medium (B = 0.212) positive effects on perceived teaching performance scores, whereas the effects of students' preparation (B = -0.055) and group size (B = -0.130) were small and negative. CONCLUSIONS: This study provides curriculum organisers and teachers indications on how to optimise variables that influence perceived teaching performance in seminars. It is suggested that teachers should search for the most appropriate combination of motivating and challenging content and facilitation method within seminars to optimise discussion opportunities between students.

Development of an instrument for measuring different types of cognitive load.

According to cognitive load theory, instructions can impose three types of cognitive load on the learner: intrinsic load, extraneous load, and germane load. Proper measurement of the different types of cognitive load can help us understand why the effectiveness and efficiency of learning environments may differ as a function of instructional formats and learner characteristics. In this article, we present a ten-item instrument for the measurement of the three types of cognitive load. Principal component analysis on data from a lecture in statistics for PhD students (n = 56) in psychology and health sciences revealed a three-component solution, consistent with the types of load that the different items were intended to measure. This solution was confirmed by a confirmatory factor analysis of data from three lectures in statistics for different cohorts of bachelor students in the social and health sciences (ns = 171, 136, and 148), and received further support from a randomized experiment with university freshmen in the health sciences (n = 58).