Does 'summative' count? The influence of the awarding of study credits on feedback use and test-taking motivation in medical progress testing.

Despite the increasing implementation of formative assessment in medical education, its' effect on learning behaviour remains questionable. This effect may depend on how students value formative, and summative assessments differently. Informed by Expectancy Value Theory, we compared test preparation, feedback use, and test-taking motivation of medical students who either took a purely formative progress test (formative PT-group) or a progress test that yielded study credits (summative PT-group). In a mixed-methods study design, we triangulated quantitative questionnaire data (n = 264), logging data of an online PT feedback system (n = 618), and qualitative interview data (n = 21) to compare feedback use, and test-taking motivation between the formative PT-group (n = 316), and the summative PT-group (n = 302). Self-reported, and actual feedback consultation was higher in the summative PT-group. Test preparation, and active feedback use were relatively low and similar in both groups. Both quantitative, and qualitative results showed that the motivation to prepare and consult feedback relates to how students value the assessment. In the interview data, a link could be made with goal orientation theory, as performance-oriented students perceived the formative PT as not important due to the lack of study credits. This led to low test-taking effort, and feedback consultation after the formative PT. In contrast, learning-oriented students valued the formative PT, and used it for self-study or self-assessment to gain feedback. Our results indicate that most students are less motivated to put effort in the test, and use feedback when there are no direct consequences. A supportive assessment environment that emphasizes recognition of the value of formative testing is required to motivate students to use feedback for learning.

Developing student-centred perspectives in PBL: how teacher profiles reveal educational needs for faculty development programmes.

BACKGROUND: In Asian higher education, PBL is not always successful, as few teachers have embraced a student-centred perspective. To cultivate such essential perspectives, faculty development programmes should address teachers' specific educational needs, which sadly is currently not sufficiently the case. This study aimed to identify teacher profiles that would reveal these specific educational needs of teachers and to investigate the relationship between these profiles and the amount of PBL training previously received. METHODS: To identify the said profiles, we performed latent profile analysis on a stratified random sample of 543 teachers based on a survey of teaching perspectives on the six aspects of Korthagen's onion model of reflection (environment, behaviour, competencies, beliefs, identity and mission). Additionally, we employed Chi-square and Mann-Whitney tests to investigate the aforementioned relationship. RESULTS: We identified six teacher profiles that resemble the diffusion of innovations theory's classification of innovation adopters: Innovators, Early adopters, Early majority 1, Early majority 2, Late majority and Laggards. The Chi-square test demonstrated that the amount of PBL training received did not differ significantly across profiles, although teachers with a more innovative profile had undergone slightly more PBL training. The Mann-Whitney test furthermore revealed for three profiles that more PBL training was associated with a higher overall score for student-centredness. When aspects were considered separately, however, this was not the case. CONCLUSIONS: The findings confirmed that current faculty development programmes are not sufficiently tailored to teachers' needs. We therefore propose that faculty development programmes be redesigned to address teachers' specific educational needs as reflected in the profiles based on the 6 aspects of the onion model. We expect such a tailored approach to more effectively promote the development of student-centred perspectives.

Assessment of teamwork in interprofessional education.

The aim of this study was to offer new insight into assessments of interprofessional education (IPE) by evaluating the association between students' self-assessment for teamwork and expert assessment of simulation and team-based writing examination. We also evaluated the relationship between students' demographic factors and their teamwork skill. A total 112 of medical and pharmacology students were divided into 36 interprofessional groups and participated interprofessional patient safety session. During the session, all students answered self-assessment for teamwork (SAT) individually. Teamwork skill of the student groups were rated by simulation assessment (SA) and team-based writing examination (WX). As a result, there was a weak correlation between the score of SAT and SA and no correlation between the score of SAT and WX. Also, as a demographic analysis, medical students assessed their teamwork higher than pharmacology students, and male students assessed their teamwork higher than female students. The duration of club activities including sports and arts correlated negatively with the score of WX. From this result, student's high self-assessment for teamwork could be an indication of good team performance in simulation assessment, but not in team-based writing examination.

The medical pause: Importance, processes and training.

Research has shown that taking 'timeouts' in medical practice improves performance and patient safety. However, the benefits of taking timeouts, or pausing, are not sufficiently acknowledged in workplaces and training programmes. To promote this acknowledgement, we suggest a systematic conceptualisation of the medical pause, focusing on its importance, processes and implementation in training programmes. By employing insights from educational and cognitive psychology, we first identified pausing as an important skill to interrupt negative momentum and bolster learning. Subsequently, we categorised constituent cognitive processes for pausing skills into two phases: the decision-making phase (determining when and how to take pauses) and the executive phase (applying relaxation or reflection during pauses). We present a model that describes how relaxation and reflection during pauses can optimise cognitive load in performance. Several strategies to implement pause training in medical curricula are proposed: intertwining pause training with training of primary skills, providing second-order scaffolding through shared control and employing auxiliary tools such as computer-based simulations with a pause function.

Extending the use of the conceptions of learning and teaching (COLT) instrument to the postgraduate setting.

BACKGROUND: Several studies have shown that conceptions of teachers on teaching and learning can influence the teaching practices and behavior in higher education. This association is also found in undergraduate medical education but not yet established in postgraduate medical setting. An instrument, Conceptions of Learning and Teaching (COLT) was developed to measure conception of teachers in undergraduate medical education. COLT is a 3-factor 18-item questionnaire. The objective of this study is to evaluate if COLT is valid for postgraduate medical education. METHODS: We invited postgraduate clinical faculty from 3 hospitals in the Netherlands to fill out the COLT. Confirmatory and exploratory factor analysis were performed to evaluate the fit of the postgraduate clinical faculty data to the COLT. Analysis of variance was done to evaluate if there was difference among the 3 hospitals in terms of the response by the clinical faculty. RESULTS: Confirmatory factor analysis showed that the postgraduate faculty data had a 2 factor structure after removal of five items. These factors were Teacher Centeredness (TC) and combined Appreciation of Active Learning and Orientation to Professional Practice (A-P) and were considered as comparable to the factors in the original COLT, expressing the post-graduate learning and teaching setting. As several items were removed, the fit was suboptimal, yet did suggest validity for use of the COLT for postgraduate medical education. CONCLUSION: The modified COLT can be used to measure conceptions of teaching and learning in postgraduate medical education. We recommend further study to improve the factor structure of the modified COLT.

Collaborative use of virtual patients after a lecture enhances learning with minimal investment of cognitive load.

BACKGROUND: The use of virtual patients (VPs), due to their high complexity and/or inappropriate sequencing with other instructional methods, might cause a high cognitive load, which hampers learning. AIM: To investigate the efficiency of instructional methods that involved three different applications of VPs combined with lectures. METHOD: From two consecutive batches, 171 out of 183 students have participated in lecture and VPs sessions. One group received a lecture session followed by a collaborative VPs learning activity (collaborative deductive). The other two groups received a lecture session and an independent VP learning activity, which either followed the lecture session (independent deductive) or preceded it (independent inductive). All groups were administrated written knowledge acquisition and retention tests as well as transfer tests using two new VPs. All participants completed a cognitive load questionnaire, which measured intrinsic, extraneous and germane load. Mixed effect analysis of cognitive load and efficiency using the R statistical program was performed. RESULTS: The highest intrinsic and extraneous load was found in the independent inductive group, while the lowest intrinsic and extraneous load was seen in the collaborative deductive group. Furthermore, comparisons showed a significantly higher efficiency, that is, higher performance in combination with lower cognitive load, for the collaborative deductive group than for the other two groups. CONCLUSION: Collaborative use of VPs after a lecture is the most efficient instructional method, of those tested, as it leads to better learning and transfer combined with lower cognitive load, when compared with independent use of VPs, either before or after the lecture.

The effectiveness of integration of virtual patients in a collaborative learning activity.

BACKGROUND: Virtual patients (VPs) have been recently integrated within different learning activities. AIM: To compare between the effect of using VPs in a collaborative learning activity and using VPs in an independent learning activity on students' knowledge acquisition, retention and transfer. METHODS: For two different topics, respectively 82 and 76 dental students participated in teaching, learning and assessment sessions with VPs. Students from a female campus and from a male campus have been randomly assigned to condition (collaborative and independent), yielding four experimental groups. Each group received a lecture followed by a learning session using two VPs per topic. Students were administrated immediate and delayed written tests as well as transfer tests using two VPs to assess their knowledge in diagnosis and treatment. RESULTS: For the treatment items of the immediate and delayed written tests, females outperformed males in the collaborative VP group but not in the independent VP group. CONCLUSION: On the female campus, the use of VPs in a collaborative learning activity is more effective than its use as an independent learning activity in enhancing students' knowledge acquisition and retention. However, the collaborative use of VPs by itself is not enough to produce consistent results across different groups of students and attention should be given to all the factors that would affect students' interaction.

The effectiveness of sequencing virtual patients with lectures in a deductive or inductive learning approach.

BACKGROUND: Virtual patients (VPs) can be sequenced with other instructional methods in different ways. AIM: To investigate the effect of sequencing VPs with lectures in a deductive approach, in comparison with an inductive approach, on students' knowledge acquisition, retention, and transfer. METHODS: For two different topics, 84 out of 87 students have participated in the lecture and VP sessions. Students from female and male campuses have been randomly assigned to one of the two learning approaches (deductive and inductive), yielding four experimental groups. Each group received a lecture session and an independent VP learning activity, which either followed the lecture session in the deductive group or preceded it in the inductive group. Students were administrated knowledge acquisition and retention written tests as well as transfer tests using two new VPs. RESULTS: There was no significant effect for the learning approach on knowledge acquisition or retention, while for knowledge transfer, males have benefited from the inductive approach in topic 1 while in the more complex topic 2, they have benefited from the deductive approach. On the other hand, females seem to be largely unaffected by learning approach. CONCLUSIONS: Sequencing VPs in inductive and deductive learning approaches leads to no significant differences on students' performance when full guidance is offered in the inductive approach.

Embedding of the progress test in an assessment program designed according to the principles of programmatic assessment.

BACKGROUND: Progress tests (PT) are used to assess students on topics from all medical disciplines. Progress testing is usually one of the assessment methods of the cognitive domain. There is limited knowledge on how positioning of the PT in a program of assessment (PoA) influences students' PT scores, use of PT feedback and perceived learning value. METHODS: We compared PT total scores and use of a PT test feedback (ProF) system in two medical courses, where the PT is either used as a summative assessment or embedded in a comprehensive PoA and used formatively. In addition, an interview study was used to explore the students' perception on use of PT feedback and perceived learning value. RESULTS: PT total scores were higher, with considerable effect sizes (ESs) and students made more use of ProF when the PT was embedded in a comprehensive PoA. Analysis of feedback in the portfolio stimulated students to look for patterns in PT results, link the PT to other assessment results, follow-up on learning objectives, and integrate the PT in their learning for the entire PoA. CONCLUSIONS: Embedding the PT in an assessment program designed according to the principles of programmatic assessment positively affects PT total scores, use of PT feedback, and perceived learning value.

Faculty development for learning and teaching of medical professionalism.

INTRODUCTION: Professionalism must be explicitly taught, but teaching professionalism is challenging, because medical teachers are not prepared to teach this content area. AIM: This study aims at designing and evaluating a faculty development programme on learning and teaching professionalism in the Arabian context. Programme development: The study used a participatory design, where four authors and 28 teachers shared the responsibility in programme design in three steps: orientation workshop for teachers, vignette development, and teaching professionalism to students. The workshop provided the cognitive base on the salient attributes of professionalism in the Arabian context. After the workshop, authors helped teachers to develop a total of 32 vignettes in various clinical aspects, portraying a blend of professionalism dilemmas. A battery of seven questions/triggers was suggested to guide students' reflection. PROGRAMME EVALUATION: The programme was evaluated with regard to its "construct" and its "outcomes". The programme has fulfilled the guiding principles for its design and it has emerged from a genuine professionalism framework from local scholarly studies in the Arabian context. Programme outcomes were evaluated at the four levels of Kirkpatrick's model; reaction, learning, behaviour, and results. DISCUSSION: The study communicates a number of context-specific issues that should be considered when teaching professionalism in Arabian culture with respect to teachers and students. Three lessons were learned from developing vignettes, as reported by the authors. This study advocates the significance of transforming faculty development from the training discourse of stand-alone interventions to mentorship paradigm of the communities of learning. CONCLUSION: A three-step approach (orientation workshop, vignettes development, and teaching professionalism) proved effective for faculty development for learning and teaching of professionalism. Professionalism can be taught using vignettes that demonstrate professionalism dilemmas in a particular context.

Understanding how residents' preferences for supervisory methods change throughout residency training: a mixed-methods study.

BACKGROUND: A major challenge for clinical supervisors is to encourage their residents to be independent without jeopardising patient safety. Residents' preferences according to level of training on this regard have not been completely explored. This study has sought to investigate which teaching methods of the Cognitive Apprenticeship (CA) model junior, intermediate and senior residents preferred and why, and how these preferences differed between groups. METHODS: We invited 301 residents of all residency programmes of Javeriana University, Bogotá, Colombia, to participate. Each resident was asked to complete a Maastricht Clinical Teaching Questionnaire (MCTQ), which, being based on the teaching methods of CA, asked residents to rate the importance to their learning of each teaching method and to indicate which of these they preferred the most and why. RESULTS: A total of 215 residents (71 %) completed the questionnaire. All concurred that all CA teaching methods were important or very important to their learning, regardless of their level of training. However, the reasons for their preferences clearly differed between groups: junior and intermediate residents preferred teaching methods that were more supervisor-directed, such as modelling and coaching, whereas senior residents preferred teaching methods that were more resident-directed, such as exploration and articulation. CONCLUSIONS: The results indicate that clinical supervision (CS) should accommodate to residents' varying degrees of development by attuning the configuration of CA teaching methods to each level of residency training. This configuration should initially vest more power in the supervisor, and gradually let the resident take charge, without ever discontinuing CS.

A Delphi study of medical professionalism in Arabian countries: the Four-Gates model.

BACKGROUND: Medical professionalism has been described as a set of attributes and behaviors, yet the Western frameworks of medical professionalism may not resonate with the cultural values of non-Western countries. AIM: This study aims to formulate a professionalism framework for healthcare providers as interpreted by local medical professionals in Arabian countries. METHODS: A purposive sample of 17 experts from diverse disciplines participated in a Delphi study in three rounds. Consensus was identified by content analysis and by numerical analysis of responses on the basic attributes of medical professionalism in Arabian context. RESULTS: Eight professional traits were shortlisted and coupled in four themes (Gates): dealing with self, dealing with tasks, dealing with others and dealing with God. Self-accountability and self-motivation were interpreted from a faithful viewpoint as "taqwa" and "ehtesab", respectively, in Arabic. DISCUSSION: The Four-Gates Model helps in better understanding of medical professionalism as grounded in the minds and culture of Arabs. The model may act as a genuine framework for teaching and learning of medical professionalism in Arab medical schools. CONCLUSION: The study highlights the divergent interpretation of medical professionalism between Western and Arabian contexts. The Four-Gates Model may work for faith-driven societies, but not for non-Muslims Arabs students or teachers or in institutions with humanistic values.

The Effects of Reflective Pauses on Performance in Simulation Training.

INTRODUCTION: The reflective pause, taking a pause during performance to reflect, is an important practice in simulation-based learning. However, for novice learners, it is a highly complex self-regulatory skill that cannot stand alone without guidance. Using educational theories, we propose how to design cognitive and metacognitive aids to guide learners with the reflective pause and investigate its effects on performance in a simulation training environment. METHODS: These effects are examined in four aspects of performance: cognitive load, primary performance, secondary performance, and encapsulation. Medical students ( N = 72) performed tasks in simulation training for emergency medicine, under 2 conditions: reflection condition ( n = 36) where reflection was prompted and guided, and control condition ( n = 36) without such reflection. RESULTS: The effects of reflective pauses emerged for 2 aspects of performance: cognitive load decreased and secondary performance improved. However, primary performance and encapsulation did not show significant difference. CONCLUSIONS: The results demonstrate that reflective pauses with cognitive and metacognitive aids implemented can enhance some aspects of performance. We suggest that to secure these effects, feedback during reflection and an adaptation period should be provided.

Medical professionalism: development and validation of the Arabian LAMPS.

AIM: The attributes of the professional physicians varies among cultures. This study aims to develop and validate a questionnaire that measures attitudes of medical students on professionalism in the Arabian context. METHOD: Thirty-two experts contributed to item generation in particular domains. The instrument was administered to Arabian medical students and interns and responses were collected using five-point Likert scales. Data were analyzed to estimate the reliability of the instrument. The inventory in its final version was labeled as the Learners' Attitude of Medical Professionalism Scale (LAMPS). RESULTS: A total of 413 medical students and interns responded from two universities in Egypt and Saudi Arabia. Means of item response ranged from 2.38 to 4.72. The highest mainly deals with "Respect to others," while the lowest belong to "Honor/Integrity." The final version of the LAMPS has 28 items in five domains, with a reliability of 0.79. DISCUSSION: The LAMPS has salient features compared to other similar instrument. It was designed based on a reliable framework in explicit behavioral items, not abstract attributes of professionalism. The LAMPS can help teachers to identify learning gaps regarding professionalism amongst their students and track attitude changes over time or as the result of interventions. CONCLUSION: To the best of our knowledge, the LAMPS is the first context-specific inventory on medical professionalism attitudes in the Arabian context.

Supervisors' transformational leadership style and residents' job crafting in surgical training: the residents' views.

Objectives: This study aims to explore the effects of three supervisors' leadership styles (transformational, transactional, and laissez-faire) on residents' job crafting. Methods: Sequential explanatory mixed-methods. First, a purposive sample of residents rated the leadership style of their supervisors and their own job crafting on the Multifactor Leadership Questionnaire and the Dutch Job Crafting Scale. The effects were tested through linear mixed effects regression analysis. Thereafter we conducted semi-structured interviews with residents and conducted a thematic analysis. Results: A total of 116 residents participated. A transformational style had a positive effect on residents' job crafting (b = .19, t(112) =3.76, p=. 009), whereas the transactional and laissez-faire styles did not. This could be explained by the fact that residents felt a positive influence of the supervisors with such style on the atmosphere for training, on the job resources available to them, and on their modelling function for how to handle the demands of the environment. Conclusions: A transformational style of the supervisor has a positive effect on residents' job crafting. Future research should explore the supervisors' perspective, as well as the effectiveness of leadership training for supervisors with a focus on resident outcomes, such as job crafting.

Relationships between medical students' co-regulatory network characteristics and self-regulated learning: a social network study.

INTRODUCTION: Recent conceptualizations of self-regulated learning acknowledge the importance of co-regulation, i.e., students' interactions with others in their networks to support self-regulation. Using a social network approach, the aim of this study is to explore relationships between characteristics of medical students' co-regulatory networks, perceived learning opportunities, and self-regulated learning. METHODS: The authors surveyed 403 undergraduate medical students during their clinical clerkships (response rate 65.5%). Using multiple regression analysis, structural equation modelling techniques, and analysis of variance, the authors explored relationships between co-regulatory network characteristics (network size, network diversity, and interaction frequency), students' perceptions of learning opportunities in the workplace setting, and self-reported self-regulated learning. RESULTS: Across all clerkships, data showed positive relationships between tie strength and self-regulated learning (ß = 0.095, p < 0.05) and between network size and tie strength (ß = 0.530, p < 0.001), and a negative relationship between network diversity and tie strength (ß = -0.474, p < 0.001). Students' perceptions of learning opportunities showed positive relationships with both self-regulated learning (ß = 0.295, p < 0.001) and co-regulatory network size (ß = 0.134, p < 0.01). Characteristics of clerkship contexts influenced both co-regulatory network characteristics (size and tie strength) and relationships between network characteristics, self-regulated learning, and students' perceptions of learning opportunities. DISCUSSION: The present study reinforces the importance of co-regulatory networks for medical students' self-regulated learning during clinical clerkships. Findings imply that supporting development of strong networks aimed at frequent co-regulatory interactions may enhance medical students' self-regulated learning in challenging clinical learning environments. Social network approaches offer promising ways of further understanding and conceptualising self- and co-regulated learning in clinical workplaces.

The effect of a three-dimensional instructional video on performance of a spatially complex procedure in surgical residents in relation to their visual-spatial abilities.

BACKGROUND: The effect of three-dimensional (3D) vs. two-dimensional (2D) video on performance of a spatially complex procedure and perceived cognitive load were examined among residents in relation to their visual-spatial abilities (VSA). METHODS: In a randomized controlled trial, 108 surgical residents performed a 5-Flap Z-plasty on a simulation model after watching the instructional video either in a 3D or 2D mode. Outcomes included perceived cognitive load measured by NASA-TLX questionnaire, task performance assessed using Observational Clinical Human Reliability Analysis and the percentage of achieved safe lengthening of the scar. RESULTS: No significant differences were found between groups. However, when accounted for VSA, safe lengthening was achieved significantly more often in the 3D group and only among individuals with high VSA (OR = 6.67, 95%CI: 1.23-35.9, p = .027). CONCLUSIONS: Overall, 3D instructional videos are as effective as 2D videos. However, they can be effectively used to enhance learning in high VSA residents.

Flexible electronic feedback using the virtues of progress testing.

The potential richness of the feedback for learners and teachers is one of the educational advantages of progress tests (PTs). Every test administration yields information on a student's knowledge level in each sub-domain of the test (cross-sectional information), and it adds a next point to the corresponding knowledge growth curve (longitudinal information). Traditional paper-based feedback has severe limitations and requires considerable effort from the learners to give meaning to the data. We reasoned that the PT data should be flexibly accessible in all pathways and with any available comparison data, according to the personal interest of the learner. For that purpose, a web-based tool (Progress test Feedback, the ProF system) was developed. This article presents the principles and features of the generated feedback and shows how it can be used. In addition to enhancement of the feedback, the ProF database of longitudinal PT-data also provides new opportunities for research on knowledge growth, and these are currently being explored.

Understanding teaching and learning conceptions among clinical faculty as a means to improve postgraduate training.

OBJECTIVES: This study aimed to determine the conceptions of teaching and learning of postgraduate medical teachers. METHODS: We invited postgraduate clinical teachers to fill out COLT (Conceptions on Learning and Teaching) questionnaire, an 18-item instrument designed to measure the conceptions of faculty in undergraduate medical education, and did a confirmatory factor analysis (CFA) to test if it was valid to be used in a postgraduate situation. Cluster analysis was done to determine different teacher profiles. We subsequently did a qualitative study among 12 clinical teachers to further explore issues related to conceptions of teaching. We used a semi-structured interview guide with vignettes summarizing five perspectives of teaching. RESULTS: Four criteria of goodness of fit indices were met, although six items had to be removed from the original COLT items. Three clusters were identified, and 51% of participants favored a transmission teaching-style perspective. For the qualitative part, three themes were identified. Majority of the teachers preferred apprenticeship and nurturing teaching-style perspective, even if they were educated through a transmission teaching-style perspective. CONCLUSIONS: Our study has shown that the COLT, although initially designed for undergraduate medical setting in the Netherlands was a valid tool in a different setting and population, with some modifications. Both the survey and the interview studies showed that the majority of the faculty chose the transmission perspective initially, but when introduced to the other perspectives, preferred apprenticeship and nurturing. The faculty readily embraced other perspectives of teaching that they believe to take into consideration the well-being of the trainees.

Development and validation of strategies to test for interoperability of virtual patients.

Interoperability of e-learning resources requires the adherence to specific standards. In the domain of virtual patients (VP) a central role is played by the MedBiquitous' MVP specification and its application profile proposed by the eViP (Electronic Virtual Patients) project. An important factor in promoting a standard is the use of metrics for assessing the conformity of the resources to the constraints imposed by the specifications. The overall aim of this study was to explore strategies to test for conformance and investigate the capabilities and limitations of automated conformance testing. A four-level scale of conformance of virtual patient packages to the eViP profile is presented, as well as two implementations of conformance testing applications. The developed tools have been tested upon level two on a sample of four VP cases acquired from the eViP repository of virtual patients.