Surveillance and screening practices of New England congenital cardiologists for patients after the Fontan operation.

INTRODUCTION: Surveillance and management guidelines for Fontan patients are lacking due to the paucity of evidence in the literature of screening efficacy on outcome measures. METHODS: The Fontan Working Group within the New England Congenital Cardiology Association designed an electronic survey to assess surveillance practices for patients with Fontan procedures among New England congenital cardiologists and to explore variability in screening low-risk vs high-risk Fontan patients across regional programs. RESULTS: Fifty-six cardiologists representing 12 regional programs responded to the survey, comprising ~40% of the total New England congenital cardiac physicians. The majority of desired testing and consultation was available within 50 miles of the patient's home institution with some limitations of cardiac catheterization and cardiac magnetic resonance imaging availability. Surveillance and screening were less frequent in low-risk Fontan patients compared to high-risk Fontan patients. Counseling practices were similar for both low-risk and high-risk Fontan patients. Aspirin monotherapy was recommended by 82% of providers for low-risk Fontan patients, while anticoagulation regimens were more varied for the high-risk population. Practitioners with ≤15 years of experience were more likely to provide quality of life testing in both low-risk and high-risk Fontan patients. There were no other major differences in testing frequencies by years of practice, quaternary vs nonquaternary care facility, or the number of Fontan patients in a practice. CONCLUSION: This survey provides insight into regional practices of screening and surveillance of Fontan patients. These data may be used to design future research studies and evidence-based guidelines to streamline the approach to manage these complex patients.

Validity Evidence for a Brief Online Key Features Examination in the Internal Medicine Clerkship.

PURPOSE: Medical educators use key features examinations (KFEs) to assess clinical decision making in many countries, but not in U.S. medical schools. The authors developed an online KFE to assess third-year medical students' decision-making abilities during internal medicine (IM) clerkships in the United States. They used Messick's unified validity framework to gather validity evidence regarding response process, internal structure, and relationship to other variables. METHOD: From February 2012 through January 2013, 759 students (at eight U.S. medical schools) had 75 minutes to complete one of four KFE forms during their IM clerkship. They also completed a survey regarding their experiences. The authors performed item analyses and generalizability studies, comparing KFE scores with prior clinical experience and National Board of Medical Examiners Subject Examination (NBME-SE) scores. RESULTS: Five hundred fifteen (67.9%) students consented to participate. Across KFE forms, mean scores ranged from 54.6% to 60.3% (standard deviation 8.4-9.6%), and Phi-coefficients ranged from 0.36 to 0.52. Adding five cases to the most reliable form would increase the Phi-coefficient to 0.59. Removing the least discriminating case from the two most reliable forms would increase the alpha coefficient to, respectively, 0.58 and 0.57. The main source of variance came from the interaction of students (nested in schools) and cases. Correlation between KFE and NBME-SE scores ranged from 0.24 to 0.47 (P < .01). CONCLUSIONS: These results provide strong evidence for response-process and relationship-to-other-variables validity and moderate internal structure validity for using a KFE to complement other assessments in U.S. IM clerkships.

Development and initial validation of an online engagement metric using virtual patients.

BACKGROUND: Considerable evidence in the learning sciences demonstrates the importance of engagement in online learning environments. The purpose of this work was to demonstrate feasibility and to develop and collect initial validity evidence for a computer-generated dynamic engagement score based on student interactions in an online learning environment, in this case virtual patients used for clinical education. METHODS: The study involved third-year medical students using virtual patient cases as a standard component of their educational program at more than 125 accredited US and Canadian medical schools. The engagement metric algorithm included four equally weighted components of student interactions with the virtual patient. We developed a self-report measure of motivational, emotional, and cognitive engagement and conducted confirmatory factor analysis to assess the validity of the survey responses. We gathered additional validity evidence through educator reviews, factor analysis of the metric, and correlations between student use of the engagement metric and self-report measures of learner engagement. RESULTS: Confirmatory factor analysis substantiated the hypothesized four-factor structure of the survey scales. Educator reviews demonstrated a high level of agreement with content and scoring cut-points (mean Pearson correlation 0.98; mean intra-class correlation 0.98). Confirmatory factor analysis yielded an acceptable fit to a one-factor model of the engagement score components. Correlations of the engagement score with self-report measures were statistically significant and in the predicted directions. CONCLUSIONS: We present initial validity evidence for a dynamic online engagement metric based on student interactions in a virtual patient case. We discuss potential uses of such an engagement metric including better understanding of student interactions with online learning, improving engagement through instructional design and interpretation of learning analytics output.

Advancing clinical reasoning in virtual patients - development and application of a conceptual framework.

Background: Clinical reasoning is a complex skill students have to acquire during their education. For educators it is difficult to explain their reasoning to students, because it is partly an automatic and unconscious process. Virtual Patients (VPs) are used to support the acquisition of clinical reasoning skills in healthcare education. However, until now it remains unclear which features or settings of VPs optimally foster clinical reasoning. Therefore, our aims were to identify key concepts of the clinical reasoning process in a qualitative approach and draw conclusions on how each concept can be enhanced to advance the learning of clinical reasoning with virtual patients. Methods: We chose a grounded theory approach to identify key categories and concepts of learning clinical reasoning and develop a framework. Throughout this process, the emerging codes were discussed with a panel of interdisciplinary experts. In a second step we applied the framework to virtual patients. Results: Based on the data we identified the core category as the "multifactorial nature of learning clinical reasoning". This category is reflected in the following five main categories: Psychological Theories, Patient-centeredness, Context, Learner-centeredness, and Teaching/Assessment. Each category encompasses between four and six related concepts. Conclusions: With our approach we were able to elaborate how key categories and concepts of clinical reasoning can be applied to virtual patients. This includes aspects such as allowing learners to access a large number of VPs with adaptable levels of complexity and feedback or emphasizing dual processing, errors, and uncertainty.

An investigation of professionalism reflected by student comments on formative virtual patient encounters.

BACKGROUND: This study explored the use of virtual patient generated data by investigating the association between students' unprofessional patient summary statements, which they entered during an on-line virtual patient case, and detection of their future unprofessional behavior. METHOD: At the USUHS, students complete a number of virtual patient encounters, including a patient summary, to meet the clerkship requirements of Internal Medicine, Family Medicine, and Pediatrics. We reviewed the summary statements of 343 students who graduated in 2012 and 2013. Each statement was rated with regard to four features: Unprofessional, Professional, Equivocal (could be construed as unprofessional), and Unanswered (students did not enter a statement). We also combined Unprofessional and Equivocal into a new category to indicate a statement receiving either rating. We then examined the associations of students' scores on these categories (i.e. whether received a particular rating or not) and Expertise score and Professionalism score reflected by a post-graduate year one (PGY-1) program director (PD) evaluation form. The PD forms contained 58 Likert-scale items designed to measure the two constructs (Expertise and Professionalism). RESULTS: The inter-rater reliability of statements coding was high (Cohen's Kappa = .97). The measure of receiving an Unprofessional or Equivocal rating was significantly correlated with lower Expertise score (r = -.19, P < .05) as well as lower Professionalism score (r = -.17, P < .05) during PGY-1. CONCLUSION: Incident reports and review of routine student evaluations are what most schools rely on to identify the majority of professionalism lapses. Unprofessionalism reflected in student entries may provide additional markers foreshadowing subsequent unprofessional behavior.

"It Feels Like Being a Real Doctor:" The Virtual Family Approach in Medical Education.

This article was migrated. The article was marked as recommended. The majority of health outcomes are determined by social determinants of health (SDOH) while medical care is responsible for as little as 20% of health outcomes. This article is an introduction to the Virtual Family (VF) approach to case based instruction; a novel strategy for addressing SDOH in medical school. The VF theoretical framework is presented and practical considerations and challenges for implementation of the VF approach at three different medical schools are offered. VFs are defined as representations of families or social groups that are not real. "Virtual," in this instance, refers to people or things that do not physically exist. The VF approach allows students and educators to adjust the "lens" of a case's focus to view the relevant determinants. The VF approach is presented as an extension of the virtual patient approach. Theoretical support for the VF approach is argued drawing on principles from modeling and simulation, effective story design, establishing a sense of human presence, serious gaming, visual design, identity leveraging, and flow theory. Challenges and benefits of the approach are described. Measures of efficacy designed to match learning goals are proposed. The VF approach is presented as practical, accessible, economical, and potentially powerful.

Using Foreign Virtual Patients With Medical Students in Germany: Are Cultural Differences Evident and Do They Impede Learning?

BACKGROUND: Learning with virtual patients (VPs) is considered useful in medical education for fostering clinical reasoning. As the authoring of VPs is highly demanding, an international exchange of cases might be desirable. However, cultural differences in foreign VPs might hamper learning success. OBJECTIVE: We investigated the need for support for using VPs from the United States at a German university, with respect to language and cultural differences. Our goal was to better understand potential implementation barriers of a intercultural VP exchange. METHODS: Two VPs were presented to 30 German medical students featuring a cultural background different from German standards with respect to diagnostic and therapeutic procedures, ethical aspects, role models, and language (as identified by a cultural adaptation framework). Participants were assigned to two groups: 14 students were advised to complete the cases without further instructions (basic group), and 16 students received written explanatory supplemental information specifically with regard to cultural differences (supplement group). Using a 6-point scale (6=strongly agree), we analyzed the results of an integrated assessment of learning success as well as an evaluation of cases by the students on usefulness for learning and potential issues regarding the language and cultural background. RESULTS: The German students found it motivating to work with cases written in English (6-point scale, 4.5 points). The clinical relevance of the VPs was clearly recognized (6 points), and the foreign language was considered a minor problem in this context (3 points). The results of the integrated learning assessment were similar in both groups (basic 53% [SD 4] vs supplement 52% [SD 4] correct answers, P=.32). However, students using the supplemental material more readily realized culturally different diagnostic and therapeutic strategies (basic 4 vs supplement 5 points, P=.39) and were less affirmative when asked about the transferability of cases to a German context (basic 5 vs supplement 3 points, P=.048). CONCLUSIONS: German students found English VPs to be highly clinically relevant, and they rated language problems much lower than they rated motivation to work on cases in English. This should encourage the intercultural exchange of VPs. The provision of supplemental explanatory material facilitates the recognition of cultural differences and might help prevent unexpected learning effects.

The Role for Virtual Patients in the Future of Medical Education.

The medical education community is working-across disciplines and across the continuum-to address the current challenges facing the medical education system and to implement strategies to improve educational outcomes. Educational technology offers the promise of addressing these important challenges in ways not previously possible. The authors propose a role for virtual patients (VPs), which they define as multimedia, screen-based interactive patient scenarios. They believe VPs offer capabilities and benefits particularly well suited to addressing the challenges facing medical education. Well-designed, interactive VP-based learning activities can promote the deep learning that is needed to handle the rapid growth in medical knowledge. Clinically oriented learning from VPs can capture intrinsic motivation and promote mastery learning. VPs can also enhance trainees' application of foundational knowledge to promote the development of clinical reasoning, the foundation of medical practice. Although not the entire solution, VPs can support competency-based education. The data created by the use of VPs can serve as the basis for multi-institutional research that will enable the medical education community both to better understand the effectiveness of educational interventions and to measure progress toward an improved system of medical education.

The Development and Preliminary Validation of a Rubric to Assess Medical Students' Written Summary Statements in Virtual Patient Cases.

PURPOSE: The ability to create a concise summary statement can be assessed as a marker for clinical reasoning. The authors describe the development and preliminary validation of a rubric to assess such summary statements. METHOD: Between November 2011 and June 2014, four researchers independently coded 50 summary statements randomly selected from a large database of medical students' summary statements in virtual patient cases to each create an assessment rubric. Through an iterative process, they created a consensus assessment rubric and applied it to 60 additional summary statements. Cronbach alpha calculations determined the internal consistency of the rubric components, intraclass correlation coefficient (ICC) calculations determined the interrater agreement, and Spearman rank-order correlations determined the correlations between rubric components. Researchers' comments describing their individual rating approaches were analyzed using content analysis. RESULTS: The final rubric included five components: factual accuracy, appropriate narrowing of the differential diagnosis, transformation of information, use of semantic qualifiers, and a global rating. Internal consistency was acceptable (Cronbach alpha 0.771). Interrater reliability for the entire rubric was acceptable (ICC 0.891; 95% confidence interval 0.859-0.917). Spearman calculations revealed a range of correlations across cases. Content analysis of the researchers' comments indicated differences in their application of the assessment rubric. CONCLUSIONS: This rubric has potential as a tool for feedback and assessment. Opportunities for future study include establishing interrater reliability with other raters and on different cases, designing training for raters to use the tool, and assessing how feedback using this rubric affects students' clinical reasoning skills.

How we developed and piloted an electronic key features examination for the internal medicine clerkship based on a US national curriculum.

BACKGROUND: Key features examinations (KFEs) have been used to assess clinical decision making in medical education, yet there are no reports of an online KFE-based on a national curriculum for the internal medicine clerkship. What we did: The authors developed and pilot tested an electronic KFE based on the US Clerkship Directors in Internal Medicine core curriculum. Teams, with expert oversight and peer review, developed key features (KFs) and cases. EVALUATION: The exam was pilot tested at eight medical schools with 162 third and fourth year medical students, of whom 96 (59.3%) responded to a survey. While most students reported that the exam was more difficult than a multiple choice question exam, 61 (83.3%) students agreed that it reflected problems seen in clinical practice and 51 (69.9%) students reported that it more accurately assessed the ability to make clinical decisions. CONCLUSIONS: The development of an electronic KFs exam is a time-intensive process. A team approach offers built-in peer review and accountability. Students, although not familiar with this format in the US, recognized it as authentically assessing clinical decision-making for problems commonly seen in the clerkship.

A collaborative model for developing and maintaining virtual patients for medical education.

There is great interest in using computer-assisted instruction in medical education, but getting computer-assisted instruction materials used broadly is difficult to achieve. We describe a successful model for the development and maintenance of a specific type of computer-assisted instruction - virtual patients - in medical education. The collaborative model's seven key components are described and compared to other models of diffusion of innovation and curriculum development. The collaborative development model that began in one medical discipline is now extended to two additional disciplines, through partnerships with their respective clerkship director organizations. We believe that the ability to achieve broad use of virtual patients, and to transition the programs from successfully relying on grant funding to financially self-sustaining, resulted directly from the collaborative development and maintenance process. This process can be used in other learning environments and for the development of other types of computer-assisted instruction programs.

Integration strategies for using virtual patients in clinical clerkships.

PURPOSE: To explore students' perceptions of virtual patient use in the clinical clerkship and develop a framework to evaluate effects of different integration strategies on students' satisfaction and perceptions of learning effectiveness with this innovation. METHOD: A prospective, multiinstitutional study was conducted at six schools' pediatric clerkships to assess the impact of integrating Web-based virtual patient cases on students' perceptions of their learning during 2004-2005 and 2005-2006. Integration strategies were designed to meet the needs of each school, and integration was scored for components of virtual patient use and elimination of other teaching methodologies. A student survey was developed, validated, and administered at the end of the clerkship to 611 students. Data were analyzed using confirmatory factor analysis and structural equation modeling. RESULTS: A total of 545 students (89%) completed the survey. Overall student satisfaction with the virtual patients was high; students reported that they were more effective than traditional methods. The structural model demonstrated that elimination of other teaching methodologies was directly associated with perceived effectiveness of the integration strategies. A higher use score had a significant negative effect on perceived integration, but a positive effect on perceived knowledge and skills gain. Students' positive perceptions of integration directly affected their satisfaction and perception of the effectiveness of their learning. CONCLUSIONS: Integration strategies balancing the use of virtual patients with elimination of some other requirements were significantly associated with students' satisfaction and their perceptions of improved knowledge and skills.

Computer-assisted instruction in clinical education: a roadmap to increasing CAI implementation.

This reflection is based on the premise that clinical education can be improved by more widespread use of computer-assisted instruction (CAI) and that a roadmap will enable more medical educators to begin using CAI. The rationale for CAI use includes many of its inherent features such as incorporation of multimedia and interactivity yet the use of CAI remains limited, apparently because educators are not convinced about the role for CAI. Barriers to CAI use are discussed including misinterpretation of the literature for CAI effectiveness; a disconnect between CAI developers and the educators who make decisions about CAI use; and the paucity of knowledge regarding how to integrate CAI effectively into clinical education. Specific roles for CAI in undergraduate and graduate medical education can include improving uniformity of instruction, providing documentation of exposure or competence, improving the learners' educational experience or outcomes, and assessment that is matched to learning. Funding for CAI remains an important barrier but the authors believe that this will be overcome when use of CAI becomes more widespread.

Multi-institutional development and utilization of a computer-assisted learning program for the pediatrics clerkship: the CLIPP Project.

Computer-assisted instruction (CAI) holds significant promise for meeting the current challenges of medical education by providing consistent and quality teaching materials regardless of training site. The Computer-assisted Learning in Pediatrics Project (CLIPP) was created over three years (2000-2003) to meet this potential through multi-institutional development of interactive Internet-based patient simulations that comprehensively teach the North American core pediatrics clerkship curriculum. Project development adhered to four objectives: (1) comprehensive coverage of the core curriculum; (2) uniform approach to CAI pedagogy; (3) multi-institutional development by educators; and (4) extensive evaluation by users. Pediatrics clerkship directors from 30 institutions worked in teams to develop a series of 31 patient case simulations. An iterative process of case content and pedagogy development, case authoring, peer review, and pilot-testing ensured that the needs of clerkship directors and medical students were met. Fifty medical schools in the United States and Canada are presently using CLIPP. More than 8,000 students have completed over 98,000 case sessions, with an average of 2,000 case sessions completed per week at this time. Each CLIPP case has been completed by more than 3,000 students. The current cost of CLIPP development is approximately $70 per student user, or $6 per case session. The project's success demonstrates that multi-institutional development and implementation of a peer-reviewed comprehensive CAI learning program by medical educators is feasible and provides a useful model for other organizations to develop similar programs. Although CAI development is both time-consuming and costly, the initial investment decreases significantly with broad use over time.

Clerkship enhancement of interpersonal skills.

BACKGROUND: The purpose of this research was to determine if students improve interpersonal skills as the third year progresses despite the lack of any specific curriculum or teaching methods. METHODS: Third-year students completed 1 of 3 16-week sequential clerkship blocks. Each student completed a clinical performance examination before and after clerkship consisting of a videotaped standardized patient interview and physical examination. Videotapes were randomly assigned to communication faculty for evaluation. RESULTS: Although the majority (73%) of students improved during their block, 17% showed no improvement, and 12% had deficient interpersonal skills after their clerkship. CONCLUSIONS: Despite the lack of skill-directed curriculum, most medical students showed improved interpersonal skill performance after a 16-week clerkship. Developing an interpersonal curriculum for all third-year students may not be necessary. Because faculty are being asked to do more with less, we believe efforts focused on individual students during the third year will be more productive.

Apical ventricular septal defects: follow-up concerning anatomic and surgical considerations.

BACKGROUND: Apical ventricular septal defects (VSDs) are difficult to visualize and close transatrially. We described their distinctive anatomic features, which have seldom been documented angiocardiographically and pathologically, in order to develop an effective approach for their surgical management. METHODS: Fourteen postmortem cases, two explanted hearts, 9 successfully operated patients, and 1 unoperated living patient were included in this report. Angiocardiographic documentation of the apical VSD was available in 14 of 16 (87.5%) of the postmortem and transplanted cases, and in 6 of 10 (60%) of the living patients. Echocardiograms were available in 23 of all 26 cases (88%). RESULTS: Severe associated malformations were present in 14 of 16 (87%) of the pathologically documented cases. Large VSDs allowed extensive communication between the left ventricular and the right ventricular sinuses in 4 patients. In 12 of the pathologically documented cases and in the 10 living patients, the left ventricular apex communicated with the right ventricular apical infundibular recess. CONCLUSIONS: Extremely large apical VSDs with severe biventricular dysplasia and dysfunction may require cardiac transplantation. Large apical VSDs can be successfully closed through a small apical infundibulotomy. This approach, applicable even in small infants, can avoid pulmonary artery banding or left ventriculotomy.