Educational assessment of the major lower limb amputations videos on YouTube.

OBJECTIVE: Videos of surgical procedures are viewed by some as potential training resources for surgeons and residents. However, there is little evidence on the effectiveness of surgical videos on learning and understanding complex three-dimensional surgical procedures. Lower extremity amputation is a complex surgery, and many residents and surgeons have low exposure to this type of procedures. This paper investigates the educational quality of lower extremity amputation videos posted on YouTube. METHODS: The search was limited to the first 100 videos. Full-length videos of any major lower limb amputation or disarticulation were included. Key basic video data such as title, YouTube address (http://), country of origin, channel source, uploading date, video duration time, number of views, number of up-voters and number of down-voters were collected. An educational assessment tool has been developed specifically for limb amputations. It consists in 11 items: three general and eight amputation-specific, each having a maximum score of 2. RESULTS: In total, 13 videos met the inclusion criteria for final analysis. Four videos reported the surgical technique of above knee amputation, two reported that of knee disarticulation and the remaining seven videos described below knee amputation. The average score (±SD) was 12.77 ± 5.2 yielding an average grade close to "Fair." A high level of concordance was found between the two assessors (κ = 0.79). No correlation was found between educational assessment tool score and the pre-set variables (r = 0.6, R2 = 35.4%, F = 1.09, P = 0.4). CONCLUSIONS: Most videos describing lower extremity amputation techniques were found to be of low-to-moderate quality. Only 4 out of 13 (30.7%) had an excellent educational and technical quality. Surgeons and surgical residents should be aware that not all posted videos on YouTube are beneficial. High educational quality videos are needed since many surgeons and residents have a low exposure to such surgeries.

Phys-MAPS: a programmatic physiology assessment for introductory and advanced undergraduates.

We describe the development of a new, freely available, online, programmatic-level assessment tool, Measuring Achievement and Progress in Science in Physiology, or Phys-MAPS ( http://cperl.lassp.cornell.edu/bio-maps ). Aligned with the conceptual frameworks of Core Principles of Physiology, and Vision and Change Core Concepts, Phys-MAPS can be used to evaluate student learning of core physiology concepts at multiple time points in an undergraduate physiology program, providing a valuable longitudinal tool to gain insight into student thinking and aid in the data-driven reform of physiology curricula. Phys-MAPS questions have a modified multiple true/false design and were developed using an iterative process, including student interviews and physiology expert review to verify scientific accuracy, appropriateness for physiology majors, and clarity. The final version of Phys-MAPS was tested with 2,600 students across 13 universities, has evidence of reliability, and has no significant statement biases. Over 90% of the physiology experts surveyed agreed that each Phys-MAPS statement was scientifically accurate and relevant to a physiology major. When testing each statement for bias, differential item functioning analysis demonstrated only a small effect size (<0.008) of any tested demographic variable. Regarding student performance, Phys-MAPS can also distinguish between lower and upper division students, both across different institutions (average overall scores increase with each level of class standing; two-way ANOVA, P < 0.001) and within each of three sample institutions (each ANOVA, P ≤ 0.001). Furthermore, at the level of individual concepts, only evolution and homeostasis do not demonstrate the typical increase across class standing, suggesting these concepts likely present consistent conceptual challenges for physiology students.

[Development of a French-language online health policy course: an international collaboration]. / Développement d'un cours francophone en ligne sur les politiques publiques en santé : une collaboration internationale.

OBJECTIVE: To present the process and challenges of developing an online competency-based course on public health policy using a collaborative international approach. METHODS: Five public health experts, supported by an expert in educational technology, adopted a rigorous approach to the development of the course: a needs analysis, identification of objectives and competencies, development of a pedagogical scenario for each module and target, choice of teaching methods and learning activities, material to be identified or developed, and the responsibilities and tasks involved. RESULTS: The 2-credit (90-hour) graduate course consists of six modules including an integration module. The modules start with a variety of case studies: tobacco law (neutral packaging), supervised injection sites, housing, integrated services for the frail elderly, a prevention programme for mothers from disadvantaged backgrounds, and the obligatory use of bicycle helmets. In modules 1, 3, 4 and 5, students learn about different stages of the public policy development process: emergence, formulation and adoption, implementation and evaluation. Module 2 focuses on the importance of values and ideologies in public policy. The integration module allows the students to apply the knowledge learned and addresses the role of experts in public policy and ethical considerations. CONCLUSION: The course has been integrated into the graduate programmes of the participating universities and allows students to follow, at a distance, an innovative training programme.

A holistic model for evaluating the impact of individual technology-enhanced learning resources.

BACKGROUND: The use of technology within education has now crossed the Rubicon; student expectations, the increasing availability of both hardware and software and the push to fully blended learning environments mean that educational institutions cannot afford to turn their backs on technology-enhanced learning (TEL). The ability to meaningfully evaluate the impact of TEL resources nevertheless remains problematic. AIMS: This paper aims to establish a robust means of evaluating individual resources and meaningfully measure their impact upon learning within the context of the program in which they are used. METHODS: Based upon the experience of developing and evaluating a range of mobile and desktop based TEL resources, this paper outlines a new four-stage evaluation process, taking into account learner satisfaction, learner gain, and the impact of a resource on both the individual and the institution in which it has been adapted. RESULTS: A new multi-level model of TEL resource evaluation is proposed, which includes a preliminary evaluation of need, learner satisfaction and gain, learner impact and institutional impact. Each of these levels are discussed in detail, and in relation to existing TEL evaluation frameworks. CONCLUSIONS: This paper details a holistic, meaningful evaluation model for individual TEL resources within the specific context in which they are used. It is proposed that this model is adopted to ensure that TEL resources are evaluated in a more meaningful and robust manner than is currently undertaken.

A Cost-Effectiveness Analysis of Blended Versus Face-to-Face Delivery of Evidence-Based Medicine to Medical Students.

BACKGROUND: Blended learning describes a combination of teaching methods, often utilizing digital technologies. Research suggests that learner outcomes can be improved through some blended learning formats. However, the cost-effectiveness of delivering blended learning is unclear. OBJECTIVE: This study aimed to determine the cost-effectiveness of a face-to-face learning and blended learning approach for evidence-based medicine training within a medical program. METHODS: The economic evaluation was conducted as part of a randomized controlled trial (RCT) comparing the evidence-based medicine (EBM) competency of medical students who participated in two different modes of education delivery. In the traditional face-to-face method, students received ten 2-hour classes. In the blended learning approach, students received the same total face-to-face hours but with different activities and additional online and mobile learning. Online activities utilized YouTube and a library guide indexing electronic databases, guides, and books. Mobile learning involved self-directed interactions with patients in their regular clinical placements. The attribution and differentiation of costs between the interventions within the RCT was measured in conjunction with measured outcomes of effectiveness. An incremental cost-effectiveness ratio was calculated comparing the ongoing operation costs of each method with the level of EBM proficiency achieved. Present value analysis was used to calculate the break-even point considering the transition cost and the difference in ongoing operation cost. RESULTS: The incremental cost-effectiveness ratio indicated that it costs 24% less to educate a student to the same level of EBM competency via the blended learning approach used in the study, when excluding transition costs. The sunk cost of approximately AUD $40,000 to transition to the blended model exceeds any savings from using the approach within the first year of its implementation; however, a break-even point is achieved within its third iteration and relative savings in the subsequent years. The sensitivity analysis indicates that approaches with higher transition costs, or staffing requirements over that of a traditional method, are likely to result in negative value propositions. CONCLUSIONS: Under the study conditions, a blended learning approach was more cost-effective to operate and resulted in improved value for the institution after the third year iteration, when compared to the traditional face-to-face model. The wider applicability of the findings are dependent on the type of blended learning utilized, staffing expertise, and educational context.

Assessment of online patient education materials from major ophthalmologic associations.

IMPORTANCE: Patients are increasingly using the Internet to supplement finding medical information, which can be complex and requires a high level of reading comprehension. Online ophthalmologic materials from major ophthalmologic associations should be written at an appropriate reading level. OBJECTIVES: To assess ophthalmologic online patient education materials (PEMs) on ophthalmologic association websites and to determine whether they are above the reading level recommended by the American Medical Association and National Institutes of Health. DESIGN, SETTING, AND PARTICIPANTS: Descriptive and correlational design. Patient education materials from major ophthalmology websites were downloaded from June 1, 2014, through June 30, 2014, and assessed for level of readability using 10 scales. The Flesch Reading Ease test, Flesch-Kincaid Grade Level, Simple Measure of Gobbledygook test, Coleman-Liau Index, Gunning Fog Index, New Fog Count, New Dale-Chall Readability Formula, FORCAST scale, Raygor Readability Estimate Graph, and Fry Readability Graph were used. Text from each article was pasted into Microsoft Word and analyzed using the software Readability Studio professional edition version 2012.1 for Windows. MAIN OUTCOMES AND MEASURES: Flesch Reading Ease score, Flesch-Kincaid Grade Level, Simple Measure of Gobbledygook grade, Coleman-Liau Index score, Gunning Fog Index score, New Fog Count, New Dale-Chall Readability Formula score, FORCAST score, Raygor Readability Estimate Graph score, and Fry Readability Graph score. RESULTS: Three hundred thirty-nine online PEMs were assessed. The mean Flesch Reading Ease score was 40.7 (range, 17.0-51.0), which correlates with a difficult level of reading. The mean readability grade levels ranged as follows: 10.4 to 12.6 for the Flesch-Kincaid Grade Level; 12.9 to 17.7 for the Simple Measure of Gobbledygook test; 11.4 to 15.8 for the Coleman-Liau Index; 12.4 to 18.7 for the Gunning Fog Index; 8.2 to 16.0 for the New Fog Count; 11.2 to 16.0 for the New Dale-Chall Readability Formula; 10.9 to 12.5 for the FORCAST scale; 11.0 to 17.0 for the Raygor Readability Estimate Graph; and 12.0 to 17.0 for the Fry Readability Graph. Analysis of variance demonstrated a significant difference (P < .001) between the websites for each reading scale. CONCLUSIONS AND RELEVANCE: Online PEMs on major ophthalmologic association websites are written well above the recommended reading level. Consideration should be given to revision of these materials to allow greater comprehension among a wider audience.

Cost-effectiveness of electronic training in domestic violence risk assessment: ODARA 101.

The need for domestic violence training has increased with the development of evidence-based risk assessment tools, which must be scored correctly for valid application. Emerging research indicates that training in domestic violence risk assessment can increase scoring accuracy, but despite the increasing popularity of electronic training, it is not yet known whether it can be an effective method of risk assessment training. In the present study, 87 assessors from various professions had training in the Ontario Domestic Assault Risk Assessment either face-to-face or using an electronic training program. The two conditions were equally effective, as measured by performance on a post-training skill acquisition test. Completion rates were 100% for face-to-face and 86% for electronic training, an improvement over a previously evaluated manual-only condition. The estimated per-trainee cost of electronic training was one third that of face-to-face training and expected to decrease. More rigorous evaluations of electronic training for risk assessment are recommended.

Filling a void: developing a standard subjective assessment tool for surgical simulation through focused review of current practices.

BACKGROUND: Simulation-based education (SBE) has advanced greatly in surgery the past decade, partly through development of validated performance measurement. Standard measures are commonly used to evaluate performance (eg, Objective Structured Assessment of Technical Skills, Fundamentals of Laparoscopic Surgery, Global Operative Assessment of Laparoscopic Skills [GOALS]). However, subjective measures are necessary for ensuring content validity during evaluation of novel simulations or simulators. Although commonly assessed, there are no standardized instruments for such surveys of participants. This lack of standardization limits the ability to compare simulations and/or simulators. We performed a focused literature review to assess current uses of subjective measures, and develop a template for a standardized assessment tool. METHODS: A representative sample of recently collected subjective measures was generated through a focused literature review of the journals Surgery, Journal of Surgical Education, and Journal of the American College of Surgeons (January 2008-November 2012) using the key words "surgical" and "simulation." RESULTS: Of the 137 articles relevant to development and/or evaluation of surgical skills curricula or simulators, 19 (12%) reported subjective measures from participants. Ten domains were identified, including Self-efficacy/Confidence/Comfort (11, 57%), Model quality/characteristics (7, 37%), Educational/Program value (6, 32%), Previous experience with procedure/simulation (5, 26%) Relevance to practice (3, 16%), Quality- trainer feedback (3, 16%), Quality-experience (2, 11%), Attitude toward specific aspects (2, 11%), Satisfaction/Enjoyability (2, 11%), Ability to perform relevant task(s) (2, 11%). Response options varied and included 4-, 5-, and 10-point rating scales, Visual Analog Scales, and open written responses. CONCLUSION: These results suggest that simulation experiences are not currently assessed in a systematic manner. However, analysis of the literature suggests that the full range of subjective measures commonly used for subjective assessment could be addressed by a unified assessment instrument. To this end, the Michigan Standard Simulation Experience Scale (MiSSES) template has been developed and is available on-line. Such a tool would provide practitioners a freely available resource used to measure performance and preferences in SBE.

Validation of a computer based objective structured clinical examination in the assessment of undergraduate dermatology courses.

Many teaching centers have now adopted objective structured clinical examination (OSCE) as an assessment method for undergraduate dermatology courses. A modification of the standard OSCE in dermatology is computer based or electronic OSCE (eOSCE). We attempted to validate the use of a computer-based OSCE in dermatology in a group of fifth year medical students. The scores of the students in the computer-based OSCE showed a strong positive correlation with the scores on the clinical presentation (Pearson's co-efficient - 0.923, P value <0.000, significant at the 0.01 level) and a good correlation with overall scores of the student (Pearson's co-efficient - 0.728, P value <0.000, significant at the 0.01 level), indicating that this is a reliable method for assessment in dermatology. Generally, the students' feedback regarding the methods was positive.

Reliability of CAD CAM technology in assessing crown preparations in a preclinical dental school environment.

In order to use CAD CAM (Computer Aided Design, Computer Aided Manufacturing) technology as an assessment tool when evaluating the preclinical performance of dental students, it is imperative that one has confidence in the reliability of the process. In this study, a variety of alignment methods were compared to determine both the consistency and accuracy of each method. Although the "Tooth Dots Diagonal" method exhibited the best precision (coefficient of variation=5.4 percent), it also represented the least accurate method when compared to the other methods tested. Using "Small Dots Diagonal" on the gingiva appears to be the best option, exhibiting an acceptable coefficient of variation (17.6 percent) and a high degree of accuracy in terms of tolerance (mean ± standard deviation=0.163 ± 0.029). Based on the results of this study, further investigation of CAD CAM technology for the purpose of assessment and education of dental students is recommended.

[Current state of competence assessment in nursing]. / Entwicklungsstand der Kompetenzmessung im Berufsfeld Pflege.

Competency measurement is central to the optimisation of outcome oriented educational processes in nursing, similar to the concept of evidence based practice. The classification of measurement tools provides the basis for describing the current state of research and development in relation to competence measurement in nursing science, and any gaps are identified. The article concludes with questioning the importance of outcome oriented quality orientation in order to achieve an increase in quality during training. Further methodological developments and qualitative studies are needed to examine the context specific processes of interaction and learning, beyond competence diagnostics.

Improving the application of a practice guideline for the assessment and treatment of suicidal behavior by training the full staff of psychiatric departments via an e-learning supported Train-the-Trainer program: study protocol for a randomized controlled trial.

BACKGROUND: In 2012, in The Netherlands a multidisciplinary practice guideline for the assessment and treatment of suicidal behavior was issued. The release of guidelines often fails to change professional behavior due to multiple barriers. Structured implementation may improve adherence to guidelines. This article describes the design of a study measuring the effect of an e-learning supported Train-the-Trainer program aiming at the training of the full staff of departments in the application of the guideline. We hypothesize that both professionals and departments will benefit from the program. METHOD: In a multicenter cluster randomized controlled trial, 43 psychiatric departments spread over 10 regional mental health institutions throughout The Netherlands will be clustered in pairs with respect to the most prevalent diagnostic category of patients and average duration of treatment. Pair members are randomly allocated to either the experimental or the control condition. In the experimental condition, the full staff of departments, that is, all registered nurses, psychologists, physicians and psychiatrists (n = 532, 21 departments) will be trained in the application of the guideline, in a one-day small interactive group Train-the-Trainer program. The program is supported by a 60-minute e-learning module with video vignettes of suicidal patients and additional instruction. In the control condition (22 departments, 404 professionals), the guideline shall be disseminated in the traditional way: through manuals, books, conferences, internet, reviews and so on. The effectiveness of the program will be assessed at the level of both health care professionals and departments. DISCUSSION: We aim to demonstrate the effect of training of the full staff of departments with an e-learning supported Train-the-Trainer program in the application of a new clinical guideline. Strengths of the study are the natural setting, the training of full staff, the random allocation to the conditions, the large scale of the study and the willingness of both staff and management to participate in the study. TRIAL REGISTRATION: Dutch trial register: NTR3092.

A quality assessment of cardiac auscultation material on YouTube.

BACKGROUND: YouTube is a highly utilized Web site that contains a large amount of medical educational material. Although some studies have assessed the education material contained on the Web site, little analysis of cardiology content has been made. This study aimed to assess the quality of videos relating to heart sounds and murmurs contained on YouTube. HYPOTHESIS: We hypothesized that the quality of video files purporting to provide education on heart auscultation would be highly variable. METHODS: Videos were searched for using the terms "heart sounds," "heart murmur," and "heart auscultation." A built-in educational filter was employed, and manual rejection of non-English language and nonrelated videos was undertaken. Remaining videos were analyzed for content, and suitable videos were scored using a purpose-built tool. RESULTS: YouTube search located 3350 videos in total, and of these, 22 were considered suitable for scoring. The average score was 4.07 out of 7 (standard deviation, 1.35). Six videos scored 5.5 or greater and 5 videos scoring 2.5 or less. There was no correlation between video score and YouTube indices of preference (hits, likes, dislikes, or search page). The quality of videos found in this study was highly variable. YouTube indications of preference were of no value in determining the value of video content. Therefore, teaching institutions or professional societies should endeavor to identify and highlight good online teaching resources. CONCLUSIONS: YouTube contains many videos relating to cardiac auscultation, but very few are valuable education resources.

[Continuing education in cattle practice - results of a survey]. / Fortbildung für die Rinderpraxis - Ergebnisse einer Umfrage.

OBJECTIVE: Continuing education is mandatory for veterinarians in Germany and Austria. The objective of this study was to analyse interests and preferences of veterinarians in cattle practice as well as to elucidate framework requirements for continuing education, including e-learning. Results should help to improve and to optimise continuing education programs. MATERIAL AND METHODS: A survey was conducted as a questionnaire via internet and shared at two local meetings as well as by email to members of the Farm Animal Health Service Styria (Tiergesundheitsdienst Steiermark). All responses were analysed anonymously. RESULTS: A total of 259 questionnaires were returned and 195 were included in the final analyses. The majority of participants (59.0%) were in farm animal practice for more than 10 years. Of the participants, 50.8% declared to have attended up to five continuing education events per year, 27.7% more than five. The majority (71.5%) had no experience with e-learning at that time. With regard to framework requirements for attending continuing education events, the majority (62.8%) of participants preferred events of 2 days over weekends. Total expenses, including costs for travelling and lodging, should not exceed 500 € per event (62.8% of participants). The favourite topics were animal reproduction (87.2%), metabolic disorders (85.6%) and mastitis (79.4%). Participants with less than 5 years of professional experience chose significantly more often the topics feed analyses, acupuncture, pregnancy diagnosis and homoeopathy/phytotherapy than participants with longer professional experience. CONCLUSION: The results of this study provide important information about the interests and framework requirements for continuing education for cattle practitioners that should help to improve the offers in continuing education programs.

Improving the efficiency of advanced life support training: a randomized, controlled trial.

BACKGROUND: Each year, more than 1.5 million health care professionals receive advanced life support (ALS) training. OBJECTIVE: To determine whether a blended approach to ALS training that includes electronic learning (e-learning) produces outcomes similar to those of conventional, instructor-led ALS training. DESIGN: Open-label, noninferiority, randomized trial. Randomization, stratified by site, was generated by Sealed Envelope (Sealed Envelope, London, United Kingdom). (International Standardized Randomized Controlled Trial Number Register: ISCRTN86380392) SETTING: 31 ALS centers in the United Kingdom and Australia. PARTICIPANTS: 3732 health care professionals recruited between December 2008 and October 2010. INTERVENTION: A 1-day course supplemented with e-learning versus a conventional 2-day course. MEASUREMENTS: The primary outcome was performance in a cardiac arrest simulation test at the end of the course. Secondary outcomes comprised knowledge- and skill-based assessments, repeated assessment after remediation training, and resource use. RESULTS: 440 of the 1843 participants randomly assigned to the blended course and 444 of the 1889 participants randomly assigned to conventional training did not attend the courses. Performance in the cardiac arrest simulation test after course attendance was lower in the electronic advanced life support (e-ALS) group compared with the conventional advanced life support (c-ALS) group; 1033 persons (74.5%) in the e-ALS group and 1146 persons (80.2%) in the c-ALS group passed (mean difference, -5.7% [95% CI, -8.8% to -2.7%]). Knowledge- and skill-based assessments were similar between groups, as was the final pass rate after remedial teaching, which was 94.2% in the e-ALS group and 96.7% in the c-ALS group (mean difference, -2.6% [CI, -4.1% to 1.2%]). Faculty, catering, and facility costs were $438 per participant for electronic ALS training and $935 for conventional ALS training. LIMITATIONS: Many professionals (24%) did not attend the courses. The effect on patient outcomes was not evaluated. CONCLUSION: Compared with conventional ALS training, an approach that included e-learning led to a slightly lower pass rate for cardiac arrest simulation tests, similar scores on a knowledge test, and reduced costs. PRIMARY FUNDING SOURCE: National Institute of Health Research and Resuscitation Council (UK).

The effect of diagnosis-specific computerized discharge instructions on 72-hour return visits to the pediatric emergency department.

OBJECTIVE: The number of patients returning to the pediatric emergency department (PED) within 72 hours of discharge is frequently cited as a benchmark for quality patient care. The purpose of this study was to determine whether the introduction of diagnosis-specific computer-generated discharge instructions would decrease the number of medically unnecessary return visits to the PED. METHODS: A retrospective chart review of patients who returned to the PED within 72 hours of discharge was performed. Charts were reviewed from 2 comparable periods: September 2004 to February 2005, when handwritten discharge instructions were issued to each patient, and September 2005 to February 2006, when each patient received computer-generated diagnosis-specific discharge instructions. The patient's age, primary care provider, insurance status, chief complaint, vital signs, history, physical examination, plan of care, and diagnosis at each visit were recorded. Cases were excluded if the patient left against medical advice or without being seen, was admitted to the hospital on the first visit, or had incomplete or missing records. The medical necessity of the return visit was rated as "yes," "no," or "indeterminate" based on review of the visit noting reason for return, history and physical examination, diagnosis, and interventions or changes in the initial care plan. RESULTS: Of all return visits to the PED within 72 hours of discharge, 13% were deemed unnecessary for patients receiving handwritten instructions compared with 15% for patients receiving computer-generated instructions (P = 0.5, not significant). For each additional year of age, the return visit was 1.07 times as likely to be medically appropriate (95% confidence interval, 1.03-1.12; P = 0.002). Patients who returned to the PED more than once were 2.69 times more likely to have a medically appropriate visit as were those with only 1 return visit (95% confidence interval, 0.95-7.58; P = 0.062). CONCLUSIONS: Computer-generated diagnosis-specific discharge instructions do not decrease the number of medically unnecessary repeat visits to the PED.

Simulation-based medical education is no better than problem-based discussions and induces misjudgment in self-assessment.

Simulation-based teaching (SBT) is increasingly used in medical education. As an alternative to other teaching methods there is a lack of evidence concerning its efficacy. The aim of this study was to evaluate the potency of SBT in anesthesia in comparison to problem-based discussion (PBD) with students in a randomized controlled setting. Thirty-three fourth-year medical students attending a curricular anesthesiology course were randomly allocated to either a session of SBT or a session of PBD on an emergency induction method. Ten days later all students underwent examination in a simulator. The performance of each student was evaluated by weighted tasks, established according to a modified Delphi process. Confidence and a multiple-choice questionnaire were additionally performed pre- and post-intervention. A total of 32 students completed the study. Participants in the SBT group presented with significantly higher self-assessment scores after the intervention than students in the PBD group. However, students in the SBT group achieved only slightly and statistically insignificantly higher scores in the theoretical and simulator examination (p > 0.05) with only a moderate effect size of d = 0.52. The current study demonstrates that both PBD and SBT lead to comparable short-term outcomes in theoretical knowledge and clinical skills. However, undesirably, SBT students overrated their anticipated clinical abilities and knowledge improvement.