The cognitive load of the in-class phase of a flipped classroom course on radiology: Could computer support be of help?

In medical education, the use of flipped classrooms is on the increase. The complex, interdependent active- and social learning activities often built into the 'in-class' phase of this new approach may, however, cause cognitive overload, impeding learning. Computer-supported collaborative learning could play a role in reducing such cognitive load. The current study investigated how four constituent factors of 'in-class' scenarios for 'flipped' classrooms affect the intrinsic, germane and extraneous cognitive load for students. After participating in a computer-supported 'in-class' scenario for a radiology training course that combined face-to-face individual activities, small-group work and class-wide activities, undergraduate medical students (n = 135) completed a questionnaire rating the cognitive load perceived. Computer support comprised a program for radiology assessment and a learning dashboard that captured and visualized the assessment results. Total intrinsic load, germane load, and extraneous load (EL) were calculated from the results obtained. The 'high' intrinsic load, 'high' germane load, and 'low' EL indicate that the computer-supported 'in-class' scenario was cognitively challenging but that sufficient mental effort could be spent on task performance and learning. We attribute the successful management of the cognitive load imposed by this complex in-class scenario to the specially designed learning dashboard.

Electronic assessment of clinical reasoning in clerkships: A mixed-methods comparison of long-menu key-feature problems with context-rich single best answer questions.

BACKGROUND: It remains unclear which item format would best suit the assessment of clinical reasoning: context-rich single best answer questions (crSBAs) or key-feature problems (KFPs). This study compared KFPs and crSBAs with respect to students' acceptance, their educational impact, and psychometric characteristics when used in a summative end-of-clinical-clerkship pediatric exam. METHODS: Fifth-year medical students (n = 377) took a computer-based exam that included 6-9 KFPs and 9-20 crSBAs which assessed their clinical reasoning skills, in addition to an objective structured clinical exam (OSCE) that assessed their clinical skills. Each KFP consisted of a case vignette and three key features using a "long-menu" question format. We explored students' perceptions of the KFPs and crSBAs in eight focus groups and analyzed statistical data of 11 exams. RESULTS: Compared to crSBAs, KFPs were perceived as more realistic and difficult, providing a greater stimulus for the intense study of clinical reasoning, and were generally well accepted. The statistical analysis revealed no difference in difficulty, but KFPs resulted more reliable and efficient than crSBAs. The correlation between the two formats was high, while KFPs correlated more closely with the OSCE score. CONCLUSIONS: KFPs with long-menu exams seem to bring about a positive educational effect without psychometric drawbacks.

Collaborative diagramming during problem based learning in medical education: Do computerized diagrams support basic science knowledge construction?

AIM: To examine how collaborative diagramming affects discussion and knowledge construction when learning complex basic science topics in medical education, including its effectiveness in the reformulation phase of problem-based learning. METHODS: Opinions and perceptions of students (n = 70) and tutors (n = 4) who used collaborative diagramming in tutorial groups were collected with a questionnaire and focus group discussions. A framework derived from the analysis of discourse in computer-supported collaborative leaning was used to construct the questionnaire. Video observations were used during the focus group discussions. RESULTS: Both students and tutors felt that collaborative diagramming positively affected discussion and knowledge construction. Students particularly appreciated that diagrams helped them to structure knowledge, to develop an overview of topics, and stimulated them to find relationships between topics. Tutors emphasized that diagramming increased interaction and enhanced the focus and detail of the discussion. Favourable conditions were the following: working with a shared whiteboard, using a diagram format that facilitated distribution, and applying half filled-in diagrams for non-content expert tutors and\or for heterogeneous groups with low achieving students. CONCLUSION: The empirical findings in this study support the findings of earlier more descriptive studies that diagramming in a collaborative setting is valuable for learning complex knowledge in medicine.

Exploring the validity and reliability of a questionnaire for evaluating virtual patient design with a special emphasis on fostering clinical reasoning.

BACKGROUND: Virtual patients (VPs) are increasingly used to train clinical reasoning. So far, no validated evaluation instruments for VP design are available. AIMS: We examined the validity of an instrument for assessing the perception of VP design by learners. METHODS: Three sources of validity evidence were examined: (i) Content was examined based on theory of clinical reasoning and an international VP expert team. (ii) The response process was explored in think-aloud pilot studies with medical students and in content analyses of free text questions accompanying each item of the instrument. (iii) Internal structure was assessed by exploratory factor analysis (EFA) and inter-rater reliability by generalizability analysis. RESULTS: Content analysis was reasonably supported by the theoretical foundation and the VP expert team. The think-aloud studies and analysis of free text comments supported the validity of the instrument. In the EFA, using 2547 student evaluations of a total of 78 VPs, a three-factor model showed a reasonable fit with the data. At least 200 student responses are needed to obtain a reliable evaluation of a VP on all three factors. CONCLUSION: The instrument has the potential to provide valid information about VP design, provided that many responses per VP are available.

Case-Based Collaborative Learning in Undergraduate Radiology Teaching-Are Essential Conditions for Group Discussions Met?

BACKGROUND: Delivering case-based collaborative learning (cCBL) at scale using technology that both presents the clinical problem authentically and seeks to foster quality group discussion is a challenge, especially argumentation which is critical for effective learning. The aim of this study was to investigate the presence of essential conditions to capitalize on a technology-enhanced cCBL scenario for teaching radiology and facilitating quality group discussion. METHODS: A questionnaire was administered to 114 fourth-year medical students who completed a technology-enhanced cCBL scenario for teaching neuroradiology. It consisted of individual online pre-class work and face-to-face in-class work, where group discussion followed individual work at a workstation. Items from the "Heedful Interrelating in Collaborative Educational Settings" scale and "positive emotional engagement" questionnaire assessed the quality of social-cognitive processes and emotional engagement during the group discussions. Structured interviews were used to explore the teachers' awareness of and engagement with the technology. RESULTS: The mean scores of most "heedfulness" items were below 3.5 (7-point scale), suggesting that participants did not enter the debriefing with a mindset conducive for argumentation. However, for the affective states "interest" and "enjoyment" the mean scores were above 5. Free text comments suggested participants enjoyed the superficial interactions, but did not necessarily engage in argumentation. Structured interviews revealed teachers were aware of the possibilities of the learning dashboard and used it as a common frame of reference, but did not really succeed to use it as a springboard for discussion. CONCLUSION: A technology-enhanced cCBL scenario is useful for teaching radiology in undergraduate medical education, but the added value of acquiring in-depth knowledge will only be achieved when students are aware of the importance of an "heedful" mind-set.

Joint online distance learning to complement postgraduate pathology training in preparation for national board examinations.

AIMS: To meet the flexible learning needs of pathology residents preparing for national board examinations, a joint distance learning approach was developed using both asynchronous and synchronous activities with whole slide images, drawing on empirical educational research on online distance learning. METHODS: In a case study of an implementation of the designed joint distance learning approach with a geographically dispersed group of pathology residents in Finland, the participants' perceptions were measured with a 12-item questionnaire covering the value of the learning opportunity, the quality of the sociocognitive processes and their emotional engagement and social cohesion. Communication during the online session was also recorded and analysed to provide objectivity to the self-report data. RESULTS: The effectiveness of joint online learning for knowledge acquisition and preparation for national board examinations was highly rated. However, despite strong emotional engagement during synchronous activities, participants reported minimal interpersonal interaction, which was also reflected in the recordings of the online session. CONCLUSION: Using a technology integration framework and guided by the principles of self-determination theory, joint distance learning is emerging as a beneficial addition to postgraduate pathology programmes in preparation for national examinations. However, to realise the full potential of interpersonal interaction, participants should be prepared for an appropriate mindset.

Learner preferences regarding integrating, sequencing and aligning virtual patients with other activities in the undergraduate medical curriculum: A focus group study.

CONTEXT: E-learning resources, such as virtual patients (VPs), can be more effective when they are integrated in the curriculum. To gain insights that can inform guidelines for the curricular integration of VPs, we explored students' perceptions of scenarios with integrated and non-integrated VPs aimed at promoting clinical reasoning skills. METHODS: During their paediatric clerkship, 116 fifth-year medical students were given at least ten VPs embedded in eight integrated scenarios and as non-integrated add-ons. The scenarios differed in the sequencing and alignment of VPs and related educational activities, tutor involvement, number of VPs, relevance to assessment and involvement of real patients. We sought students' perceptions on the VP scenarios in focus group interviews with eight groups of 4-7 randomly selected students (n = 39). The interviews were recorded, transcribed and analysed qualitatively. RESULTS: The analysis resulted in six themes reflecting students' perceptions of important features for effective curricular integration of VPs: (i) continuous and stable online access, (ii) increasing complexity, adapted to students' knowledge, (iii) VP-related workload offset by elimination of other activities, (iv) optimal sequencing (e.g.: lecture--1 to 2 VP(s)--tutor-led small group discussion--real patient) and (V) optimal alignment of VPs and educational activities, (vi) inclusion of VP topics in assessment. CONCLUSIONS: The themes appear to offer starting points for the development of a framework to guide the curricular integration of VPs. Their impact needs to be confirmed by studies using quantitative controlled designs.

Computer supported collaborative learning in a clerkship: an exploratory study on the relation of discussion activity and revision of critical appraisal papers.

BACKGROUND: Medical students in clerkship are continuously confronted with real and relevant patient problems. To support clinical problem solving skills, students perform a Critical Appraisal of a Topic (CAT) task, often resulting in a paper. Because such a paper may contain errors, students could profit from discussion with peers, leading to paper revision. Active peer discussion by a Computer Supported Collaborative Learning (CSCL) environment show positive medical students perceptions on subjective knowledge improvement. High students' activity during discussions in a CSCL environment demonstrated higher task-focussed discussion reflecting higher levels of knowledge construction. However, it remains unclear whether high discussion activity influences students' decisions revise their CAT paper. The aim of this research is to examine whether students who revise their critical appraisal papers after discussion in a CSCL environment show more task-focussed activity and discuss more intensively on critical appraisal topics than students who do not revise their papers. METHODS: Forty-seven medical students, stratified in subgroups, participated in a structured asynchronous online discussion of individual written CAT papers on self-selected clinical problems. The discussion was structured by three critical appraisal topics. After the discussion, the students could revise their paper. For analysis purposes, all students' postings were blinded and analysed by the investigator, unaware of students characteristics and whether or not the paper was revised. Postings were counted and analysed by an independent rater, Postings were assigned into outside activity, non-task-focussed activity or task-focussed activity. Additionally, postings were assigned to one of the three critical appraisal topics. Analysis results were compared by revised and unrevised papers. RESULTS: Twenty-four papers (51.6%) were revised after the online discussion. The discussions of the revised papers showed significantly higher numbers of postings, more task-focussed activities, and more postings about the two critical appraisal topics: "appraisal of the selected article(s)", and "relevant conclusion regarding the clinical problem". CONCLUSION: A CSCL environment can support medical students in the execution and critical appraisal of authentic tasks in the clinical workplace. Revision of CAT papers appears to be related to discussions activity, more specifically reflecting high task-focussed activity of critical appraisal topics.

Design principles for virtual patients: a focus group study among students.

OBJECTIVES: This study aimed to examine what students perceive as the ideal features of virtual patient (VP) design in order to foster learning with a special focus on clinical reasoning. METHODS: A total of 104 Year 5 medical students worked through at least eight VPs representing four different designs during their paediatric clerkship. The VPs were presented in two modes and differed in terms of the authenticity of the user interface (with or without graphics support), predominant question type (long- versus short-menu questions) and freedom of navigation (relatively free versus predetermined). Each mode was presented in a rich and a poor version with regard to the use of different media and questions and explanations explicitly directed at clinical reasoning. Five groups of between four and nine randomly selected students (n = 27) participated in focus group interviews facilitated by a moderator using a questioning route. The interviews were videotaped, transcribed and analysed. Summary reports were approved by the students. RESULTS: Ten principles of VP design emerged from the analysis. A VP should be relevant, of an appropriate level of difficulty, highly interactive, offer specific feedback, make optimal use of media, help students focus on relevant learning points, offer recapitulation of key learning points, provide an authentic web-based interface and student tasks, and contain questions and explanations tailored to the clinical reasoning process. CONCLUSIONS: Students perceived the design principles identified as being conducive to their learning. Many of these principles are supported by the results of other published studies. Future studies should address the effects of these principles using quantitative controlled designs.

Towards a typology of virtual patients.

INTRODUCTION: Although on-screen "virtual patients (VPs)" have been around for decades it is only now that they are entering the mainstream, and as such they are new to most of the medical education community. There is significant variety in the form, function, and efficacy of different VPs and there is, therefore, a growing need to clarify and distinguish between them. This article seeks to clarify VP concepts and approaches using a typology of VP designs. METHODS: The authors developed a VP design typology based on the literature, a review of existing VP systems, and their personal experience with VPs. This draft framework was refined using a Delphi study involving experts in the field, and was then validated by applying it in the description of different VP designs. RESULTS: Nineteen factors were synthesized around four categories: general (title, description, language, identifier, provenance, and typical study time); educational (educational level, educational modes, coverage, and objectives); instructional design (path type, user modality, media use, narrative use, interactivity use, and feedback use); technical (originating system, format, integration, and dependence). CONCLUSION: This empirically derived VP design typology provides a common reference point for all those wishing to report on or study VPs.

The effect of face-to-face collaborative learning on the elaboration of computer-based simulated cases.

INTRODUCTION: This study investigates the effects of working face to face in small groups on the processes that occur when students elaborate on computer-based simulated cases. METHODS: We performed a randomized controlled experiment that was designed to measure the effect of "social context" (triads versus individuals) on students' perceptions of the elaboration process and on the time spent on the different parts of the computer case. We sought students' perceptions using a questionnaire that was administered to all participating students (N = 47) and we examined the actions of the students working in triads (N = 12) and individually (N = 11) by analyzing the log files of the computer case. RESULTS: The results demonstrated no significant effect of social context on the degree of elaboration of the computer case. CONCLUSIONS: Working with computer-based simulated cases in small groups as opposed to individually in itself is not enough to increase the scope and depth of the elaboration of computer cases.

Student perceptions of a virtual learning environment for a problem-based learning undergraduate medical curriculum.

OBJECTIVE: To investigate the effects of a virtual learning environment (VLE) on group interaction and consultation of information resources during the preliminary phase, self-study phase and reporting phase of the problem-based learning process in an undergraduate medical curriculum. METHODS: A questionnaire was administered to 355 medical students in Years 1 and 2 to ask them about the perceived usefulness of a virtual learning environment that was created with Blackboard for group interaction and the use of learning resources. RESULTS: The students indicated that the VLE supported face-to-face interaction in the preliminary discussion and in the reporting phase but did not stimulate computer-mediated distance interaction during the self-study phase. They perceived that the use of multimedia in case presentations led to a better quality of group discussion than if case presentations were exclusively text-based. They also indicated that the information resources that were hyperlinked in the VLE stimulated the consultation of these resources during self-study, but not during the reporting phase. CONCLUSIONS: Students indicated that the use of a VLE in the tutorial room and the inclusion of multimedia in case presentations supported processes of active learning in the tutorial groups. However, if we want to exploit the full potential of asynchronous computer-mediated communication to initiate in-depth discussion during the self-study phase, its application will have to be selective and deliberate. Students indicated that the links in the VLE to selected information in library repositories supported their learning.

Analysis of verbal interactions in tutorial groups: a process study.

INTRODUCTION: Collaborative learning, including problem-based learning (PBL), is a powerful learning method. Group interaction plays a crucial role in stimulating student learning. However, few studies on learning processes in medical education have examined group interactions. Most studies on collaboration within PBL used self-reported data rather than observational data. We investigated the following types of interactions in PBL tutorial groups: learning-oriented interactions (exploratory questioning, cumulative reasoning and handling conflicts about knowledge); procedural interactions, and irrelevant/off-task interactions. AIM: The central question concerned how much time is spent on the different types of interaction during group sessions and how the types of interaction are distributed over the meeting. METHOD: Four tutorial group sessions in Year 2 of the PBL undergraduate curriculum of Maastricht Medical School were videotaped and analysed. The sessions concerned the reporting phase of the PBL process. We analysed the interactions using a coding scheme distinguishing several verbal interaction types, such as questions, arguments and evaluations. RESULTS: Learning-orientated interactions accounted for 80% of the interactions, with cumulative reasoning, exploratory questioning and handling conflicts about knowledge accounting for about 63%, 10% and 7% of the interactions, respectively. Exploratory questioning often preceded cumulative reasoning. Both types occurred throughout the meeting. Handling conflicts mainly occurred after the first 20 minutes. CONCLUSIONS: Task involvement in the tutorial groups was high. All types of learning-orientated interactions were observed. Relatively little time was spent on exploratory questions and handling conflicts about knowledge. Problem-based learning training should pay special attention to stimulating discussion about contradictory information.