Electronic assessment of clinical reasoning in clerkships: A mixed-methods comparison of long-menu key-feature problems with context-rich single best answer questions.

BACKGROUND: It remains unclear which item format would best suit the assessment of clinical reasoning: context-rich single best answer questions (crSBAs) or key-feature problems (KFPs). This study compared KFPs and crSBAs with respect to students' acceptance, their educational impact, and psychometric characteristics when used in a summative end-of-clinical-clerkship pediatric exam. METHODS: Fifth-year medical students (n = 377) took a computer-based exam that included 6-9 KFPs and 9-20 crSBAs which assessed their clinical reasoning skills, in addition to an objective structured clinical exam (OSCE) that assessed their clinical skills. Each KFP consisted of a case vignette and three key features using a "long-menu" question format. We explored students' perceptions of the KFPs and crSBAs in eight focus groups and analyzed statistical data of 11 exams. RESULTS: Compared to crSBAs, KFPs were perceived as more realistic and difficult, providing a greater stimulus for the intense study of clinical reasoning, and were generally well accepted. The statistical analysis revealed no difference in difficulty, but KFPs resulted more reliable and efficient than crSBAs. The correlation between the two formats was high, while KFPs correlated more closely with the OSCE score. CONCLUSIONS: KFPs with long-menu exams seem to bring about a positive educational effect without psychometric drawbacks.

A Pipeline for Real-Time Synchronization of Relevant Data in Pediatric Emergencies.

Pediatric emergencies are rare events that require precise medical intervention. The dosing of medication is particularly tricky. The challenges require technical aids for automatic dosage calculation without restricting the emergency physicians' ability to act. This short paper describes the development process of an automatic data processing pipeline that synchronizes multiple sensor data to perform the necessary calculations. In a final evaluation process, two transmission protocols (HTTP and MQTT)), are compared to determine which is more suitable for this application. The results of the evaluation clearly showed that MQTT is the preferred choice for the data processing pipeline due to its significantly lower latency.

[OPTINOFA-Intelligent assistance service for structured assessment in the emergency department]. / OPTINOFA ­ Intelligenter Assistenzdienst zur strukturierten Ersteinschätzung in der Notaufnahme.

BACKGROUND: Case numbers in central emergency departments (EDs) have risen during the past decade in Germany, leading to recurrent overcrowding, increased risks in emergency care, and elevated costs. Particularly the fraction of outpatient emergency treatments has increased disproportionately. Within the framework of the Optimization of emergency care by structured triage with intelligent assistant service (OPTINOFA, Förderkennzeichen [FKZ] 01NVF17035) project, an intelligent assistance service was developed. PATIENTS AND METHODS: New triage algorithms were developed for the 20 most frequent leading symptoms on the basis of established triage systems (emergency severity index, ESI; Manchester triage system, MTS) and provided as web-based intelligent assistance services on mobile devices. To evaluate the validity, reliability, and safety of the new OPTINOFA triage instrument, a pilot study was conducted in three EDs after ethics committee approval. RESULTS: In the pilot study, n = 718 ED patients were included (age 59.1 ± 22 years; 349 male, 369 female). With respect to disposition (out-/inpatient), a sensitivity of 91.1% and a specificity of 40.7%, and a good correlation with the OPTINOFA triage levels were detected (Spearman's rank correlation ρ = 0.41). Furthermore, the area under the curve (AUC) for prediction of disposition according to the OPTINOFA triage level was 0.73. The in-hospital mortality rate of OPTINOFA triage levels 4 and 5 was 0%. The association between the length of ED stay and the OPTINOFA triage level was shown to be significant (p < 0.001). CONCLUSION: The results of the pilot study demonstrate the safety and validity of the new triage system OPTINOFA. By definition of both urgency and emergency care level, new customized perspectives for load reduction in German EDs via a closer cooperation between out- and inpatient sectors of emergency care could be established.

Learner preferences regarding integrating, sequencing and aligning virtual patients with other activities in the undergraduate medical curriculum: A focus group study.

CONTEXT: E-learning resources, such as virtual patients (VPs), can be more effective when they are integrated in the curriculum. To gain insights that can inform guidelines for the curricular integration of VPs, we explored students' perceptions of scenarios with integrated and non-integrated VPs aimed at promoting clinical reasoning skills. METHODS: During their paediatric clerkship, 116 fifth-year medical students were given at least ten VPs embedded in eight integrated scenarios and as non-integrated add-ons. The scenarios differed in the sequencing and alignment of VPs and related educational activities, tutor involvement, number of VPs, relevance to assessment and involvement of real patients. We sought students' perceptions on the VP scenarios in focus group interviews with eight groups of 4-7 randomly selected students (n = 39). The interviews were recorded, transcribed and analysed qualitatively. RESULTS: The analysis resulted in six themes reflecting students' perceptions of important features for effective curricular integration of VPs: (i) continuous and stable online access, (ii) increasing complexity, adapted to students' knowledge, (iii) VP-related workload offset by elimination of other activities, (iv) optimal sequencing (e.g.: lecture--1 to 2 VP(s)--tutor-led small group discussion--real patient) and (V) optimal alignment of VPs and educational activities, (vi) inclusion of VP topics in assessment. CONCLUSIONS: The themes appear to offer starting points for the development of a framework to guide the curricular integration of VPs. Their impact needs to be confirmed by studies using quantitative controlled designs.

Proposal of a Method for Transferring High-Quality Scientific Literature Data to Virtual Patient Cases Using Categorical Data Generated by Bernoulli-Distributed Random Values: Development and Prototypical Implementation.

BACKGROUND: Teaching medicine is a complex task because medical teachers are also involved in clinical practice and research and the availability of cases with rare diseases is very restricted. Automatic creation of virtual patient cases would be a great benefit, saving time and providing a wider choice of virtual patient cases for student training. OBJECTIVE: This study explored whether the medical literature provides usable quantifiable information on rare diseases. The study implemented a computerized method that simulates basic clinical patient cases utilizing probabilities of symptom occurrence for a disease. METHODS: Medical literature was searched for suitable rare diseases and the required information on the respective probabilities of specific symptoms. We developed a statistical script that delivers basic virtual patient cases with random symptom complexes generated by Bernoulli experiments, according to probabilities reported in the literature. The number of runs and thus the number of patient cases generated are arbitrary. RESULTS: We illustrated the function of our generator with the exemplary diagnosis "brain abscess" with the related symptoms "headache, mental status change, focal neurologic deficit, fever, seizure, nausea and vomiting, nuchal rigidity, and papilledema" and the respective probabilities from the literature. With a growing number of repetitions of the Bernoulli experiment, the relative frequencies of occurrence increasingly converged with the probabilities from the literature. For example, the relative frequency for headache after 10.000 repetitions was 0.7267 and, after rounding, equaled the mean value of the probability range of 0.73 reported in the literature. The same applied to the other symptoms. CONCLUSIONS: The medical literature provides specific information on characteristics of rare diseases that can be transferred to probabilities. The results of our computerized method suggest that automated creation of virtual patient cases based on these probabilities is possible. With additional information provided in the literature, an extension of the generator can be implemented in further research.

Development of a Clinical Decision Support System for Smart Algorithms in Emergency Medicine.

The development of clinical decision support systems (CDSS) is complex and requires user-centered planning of assistive interventions. Especially in the setting of emergency care requiring time-critical decisions and interventions, it is important to adapt a CDSS to the needs of the user in terms of acceptance, usability and utility. In the so-called ENSURE project, a user-centered approach was applied to develop the CDSS intervention. In the context of this paper, we present a path to the first mockup development for a CDSS interface by addressing Campbell's Five Rights within the CDSS workflow.

Towards a Ubiquitous Child Emergency App - Ideas to Simplify and Ensure the Machine-2-Machine Communication.

Medical emergencies involving children are rare events. The experience of emergency physicians is therefore low and the results are correspondingly poor. Assistance services to help in emergencies are regularly requested. The use case is thus very complicated, a complex system consisting of multiple devices is necessary to provide the most efficient and effective service. This short paper presents prototypically tested ideas on how such a ubiquitous approach can be designed and how communication between devices can be simplified and ensured.

Automated Size Recognition in Pediatric Emergencies Using Machine Learning and Augmented Reality: Within-Group Comparative Study.

BACKGROUND: Pediatric emergencies involving children are rare events, and the experience of emergency physicians and the results of such emergencies are accordingly poor. Anatomical peculiarities and individual adjustments make treatment during pediatric emergency susceptible to error. Critical mistakes especially occur in the calculation of weight-based drug doses. Accordingly, the need for a ubiquitous assistance service that can, for example, automate dose calculation is high. However, few approaches exist due to the complexity of the problem. OBJECTIVE: Technically, an assistance service is possible, among other approaches, with an app that uses a depth camera that is integrated in smartphones or head-mounted displays to provide a 3D understanding of the environment. The goal of this study was to automate this technology as much as possible to develop and statistically evaluate an assistance service that does not have significantly worse measurement performance than an emergency ruler (the state of the art). METHODS: An assistance service was developed that uses machine learning to recognize patients and then automatically determines their size. Based on the size, the weight is automatically derived, and the dosages are calculated and presented to the physician. To evaluate the app, a small within-group design study was conducted with 17 children, who were each measured with the app installed on a smartphone with a built-in depth camera and a state-of-the-art emergency ruler. RESULTS: According to the statistical results (one-sample t test; P=.42; α=.05), there is no significant difference between the measurement performance of the app and an emergency ruler under the test conditions (indoor, daylight). The newly developed measurement method is thus not technically inferior to the established one in terms of accuracy. CONCLUSIONS: An assistance service with an integrated augmented reality emergency ruler is technically possible, although some groundwork is still needed. The results of this study clear the way for further research, for example, usability testing.

Mobile Robotic Telepresence Between Hospital and School: Lessons Learned.

If a pupil falls seriously ill, it is not only a shock for the pupil himself or herself, but also for his or her family and classmates. The project "Virtual Classroom" of the Heilbronn University in cooperation with the foundation "Big Help for Little Heroes" therefore tries to maintain the daily school routine and thus the contact with friends as far as possible with the help of mobile telepresence robots in the classroom from home or from the hospital. Here there are both technical and human factors that contribute to success or failure. Based on a systematic literature research and practical experience, these factors are identified, discussed and weighted in this paper. If the mobile telepresence robot is used successfully, this pays off twice for the pupils affected: It has been shown that pupils can progress socially as well as in school, even if attendance at school is not possible for a long time.

Digital teaching as an instrument for cross-location teaching networks in medical informatics: opportunities and challenges.

The increasingly digitized healthcare system requires new skills from all those involved. In order to impart these competencies, appropriate courses must be developed at educational institutions. In view of the rapid development of new aspects of digitization, this presents a challenge; suitable teaching formats must be developed successively. The establishment of cross-location teaching networks is one way to better meet training needs and to make the necessary spectrum of educational content available. As part of the Medical Informatics Initiative, the HiGHmed consortium is establishing such a teaching network, in the field of medical informatics, which covers many topics related to the digitization of the health care system. Various problem areas in the German education system were identified that hinder the development of the teaching network. These problem areas were prioritized firstly according to the urgency of the solution from the point of view of the HiGHmed consortium and secondly according to existing competencies in the participating societies. A workshop on the four most relevant topics was organized with experts from the German Society for Medical Informatics, Biometry and Epidemiology (GMDS), the Society for Medical Education (GMA) and the HiGHmed consortium. These are: recognition of exam results from teaching modules that are offered digitally and across locations, and their integration into existing curricula; recognition of digital, cross-location teaching in the teachers' teaching load; nationwide uniform competencies for teachers, in order to be able to conduct digital teaching effectively and with comparable quality; technical infrastructure to efficiently and securely communicate and manage the recognition of exam results between educational institutions. For all subject areas, existing preliminary work was identified on the basis of working questions, and short- and long-term needs for action were formulated. Finally, a need for the redesign of a technologically supported syntactic and semantic interoperability of learning performance recording was identified.

Design principles for virtual patients: a focus group study among students.

OBJECTIVES: This study aimed to examine what students perceive as the ideal features of virtual patient (VP) design in order to foster learning with a special focus on clinical reasoning. METHODS: A total of 104 Year 5 medical students worked through at least eight VPs representing four different designs during their paediatric clerkship. The VPs were presented in two modes and differed in terms of the authenticity of the user interface (with or without graphics support), predominant question type (long- versus short-menu questions) and freedom of navigation (relatively free versus predetermined). Each mode was presented in a rich and a poor version with regard to the use of different media and questions and explanations explicitly directed at clinical reasoning. Five groups of between four and nine randomly selected students (n = 27) participated in focus group interviews facilitated by a moderator using a questioning route. The interviews were videotaped, transcribed and analysed. Summary reports were approved by the students. RESULTS: Ten principles of VP design emerged from the analysis. A VP should be relevant, of an appropriate level of difficulty, highly interactive, offer specific feedback, make optimal use of media, help students focus on relevant learning points, offer recapitulation of key learning points, provide an authentic web-based interface and student tasks, and contain questions and explanations tailored to the clinical reasoning process. CONCLUSIONS: Students perceived the design principles identified as being conducive to their learning. Many of these principles are supported by the results of other published studies. Future studies should address the effects of these principles using quantitative controlled designs.

Evaluation of Chatbot Prototypes for Taking the Virtual Patient's History.

In medical education Virtual Patients (VP) are often applied to train students in different scenarios such as recording the patient's medical history or deciding a treatment option. Usually, such interactions are predefined by software logic and databases following strict rules. At this point, Natural Language Processing/Machine Learning (NLP/ML) algorithms could help to increase the overall flexibility, since most of the rules can derive directly from training data. This would allow a more sophisticated and individual conversation between student and VP. One type of technology that is heavily based on such algorithmic advances are chatbots or conversational agents. Therefore, a literature review is carried out to give insight into existing educational ideas with such agents. Besides, different prototypes are implemented for the scenario of taking the patient's medical history, responding with the classified intent of a generic anamnestic question. Although the small number of questions (n=109) leads to a high SD during evaluation, all scores (recall, precision, f1) reach already a level above 80% (micro-averaged). This shows a first promising step to use these prototypes for taking the medical history of a VP.

Evaluation of Depth Cameras for Use as an Augmented Reality Emergency Ruler.

Children are rarely affected by medical emergencies. The experience of doctors or paramedics with child emergencies is correspondingly poor. The anatomical features and individual calculations make such an emergency much more error-prone than a comparable adult emergency. Particularly in dose calculations, critical errors occur time and again. Since these calculations are based on the child's weight, which is preclinically often derived from the size of the child, the number of errors can be minimized with an assistance service that performs all calculations based on the size. Technically, it is possible to detect the size with a depth camera, which is occasionally installed in smartphones or head-mounted displays. In order to investigate to what extent these cameras provide precise results, a study with 33 children was carried out. The children were measured with both an emergency ruler and an augmented reality app with associated smartphone with depth camera. The result is that the depth camera does not provide significantly different results than an emergency ruler. This allows further research, e.g. the automatic recognition of patients with the help of machine learning or usability studies, to be tackled.

Speech Recognition Evaluation of a State-Of-The-Art Smartglass in the Use Case Pediatric Resuscitation.

Emergencies involving children are rare events. Due to the associated lack of routine and special features in pediatric resuscitation, it is prone to errors and the results are unsatisfactory. One way of tackling this problem is to use assistance services. However, due to the process, these services cannot be easily integrated. One possibility is the use of Head Mounted Displays. These are often controlled via voice commands. With medical terms, the voice control implemented as standard can quickly become unusable. A wearable app was therefore developed for this paper and evaluated according to ISO/IEC 30122-2:2017 to determine the extent to which the voice control of a state-of-the-art smartglass works in quiet and noisy conditions for use during a resuscitation. Since the commands were well understood and the app could be reliably controlled, the use of voice control in an assistance service is conceivable.

Digital Teaching and Digital Medicine: A national initiative is needed.

The advancing digitization of the healthcare system requires that in the future digital skills should be communicated to students much more. In addition, digital teaching and learning technologies should be used wherever they offer real benefits over other training scenarios. To meet these challenges, it needs a national initiative "Medical Education in the Digital Age", which should be led by the Society for Medical Education and the German Association for Medical Informatics, Biometry and Epidemiology.

Can design principles of traditional learning theories be fulfilled by computer-based training systems in medicine: the example of CAMPUS.

PURPOSE: Computer-based training (CBT) systems offer the potential to efficiently support modern teaching and learning. However, it is still unknown if a similar efficient learning experience built on sound learning theories and corresponding design principles can be created in the complex health care environment. The purpose of this paper is to analyse to what extent learning theories and corresponding design principles are relevant and can successfully be applied in computer-based training in medicine. METHODS: We use the case-based CBT system CAMPUS as an example for a CBT system currently used to enhance the medical teaching and learning experience. We apply two well-accepted learning theories (Bloom's taxonomy and practice fields) and related design principles to determine to what extent they are relevant and fulfilled in the context of CAMPUS. RESULTS: We demonstrate that in principle these learning theories and design principles can be implemented using computer-based training. However, not all design principles can be fulfilled by the system alone; rather the integration of the system into adequate -- traditional or virtual -- teaching and learning environments is essential. CONCLUSIONS: Traditional learning theories and design principles are a valuable means in designing adequate CBT systems in medicine. They can be successfully implemented in CBT systems for medical education if the system itself is adequately integrated into teaching and learning environments.

Evaluation of Game Engines for Cross-Platform Development of Mobile Serious Games for Health.

Studies have shown that serious games for health can improve patient compliance and help to increase the quality of medical education. Due to a growing availability of mobile devices, especially the development of cross-platform mobile apps is helpful for improving healthcare. As the development can be highly time-consuming and expensive, an alternative development process is needed. Game engines are expected to simplify this process. Therefore, this article examines the question whether using game engines for cross-platform serious games for health can simplify the development compared to the development of a plain HTML5 app. At first, a systematic review of the literature was conducted in different databases (MEDLINE, ACM and IEEE). Afterwards three different game engines were chosen, evaluated in different categories and compared to the development of a HTML5 app. This was realized by implementing a prototypical application in the different engines and conducting a utility analysis. The evaluation shows that the Marmalade engine is the best choice for development in this scenario. Furthermore, it is obvious that the game engines have great benefits against plain HTML5 development as they provide components for graphics, physics, sounds, etc. The authors recommend to use the Marmalade Engine for a cross-platform mobile Serious Game for Health.

Wearable Technology in Medicine: Machine-to-Machine (M2M) Communication in Distributed Systems.

Smart wearables are capable of supporting physicians during various processes in medical emergencies. Nevertheless, it is almost impossible to operate several computers without neglecting a patient's treatment. Thus, it is necessary to set up a distributed network consisting of two or more computers to exchange data or initiate remote procedure calls (RPC). If it is not possible to create flawless connections between those devices, it is not possible to transfer medically relevant data to the most suitable device, as well as to control a device with another one. This paper shows how wearables can be paired and what problems occur when trying to pair several wearables. Furthermore, it is described as to what interesting scenarios are possible in the context of emergency medicine/paramedicine.

CAMPUS--a flexible, interactive system for web-based, problem-based learning in health care.

Care for patients is the best way to learn medicine and medical methods and skills. But the availability of real patients for learners is limited. Often the appropriate patient is missed or he can not be demonstrated to all students because of practical or ethical problems. A possible solution is the presentation of a clinical problem by a virtual patient using a simulative computer program. But such an approach means often a lot of work for the author of the case. We developed a flexible and realistic system, CAMPUS (www.medicase.de), which supports the case authors with appropriate vocabularies and a comfortable authoring tool. There are different kinds of case presentations to the users in accordance to the level of professionalism of the user and the scenario the program is used in. An evaluation within an internship has shown, that such pro-grams are judged as useful by medical students. But further studies are necessary to examine if the program will be used for the self-study and to develop further kinds of using scenarios within problem-based curricula.

Virtual patients feedback system--a concept to support students and educators.

Assessments deliver information about the knowledge level of a student. Formative assessments' main purposes are to identify student's weaknesses and strengths, and support educators in the planning process of their instruction. In this paper a graphical user interface concept is presented to provide feedback with the aid of the editing results of medical students, who train with Virtual Patient, which is a computer-based simulation of patient care.