Comparing Entrustment and Competence: An Exploratory Look at Performance-Relevant Information in the Final Year of a Veterinary Program.

Workplace-based assessments and entrustment scales have two primary goals: providing formative information to assist students with future learning; and, determining if and when learners are ready for safe, independent practice. To date, there has not been an evaluation of the relationship between these performance-relevant information pieces in veterinary medicine. This study collected quantitative and qualitative data from a single cohort of final-year students (n = 27) across in-training evaluation reports (ITERs) and entrustment scales in a distributed veterinary hospital environment. Here we compare progression in scoring and performance within and across student, within and across method of assessment, over time. Narrative comments were quantified using the Completed Clinical Evaluation Report Rating (CCERR) instrument to assess quality of written comments. Preliminary evidence suggests that we may be capturing different aspects of performance using these two different methods. Specifically, entrustment scale scores significantly increased over time, while ITER scores did not. Typically, comments on entrustment scale scores were more learner specific, longer, and used more of a coaching voice. Longitudinal evaluation of learner performance is important for learning and demonstration of competence; however, the method of data collection could influence how feedback is structured and how performance is ultimately judged.

Video Recording in Veterinary Medicine OSCEs: Feasibility and Inter-rater Agreement between Live Performance Examiners and Video Recording Reviewing Examiners.

The Objective Structured Clinical Examination (OSCE) is a valid, reliable assessment of veterinary students' clinical skills that requires significant examiner training and scoring time. This article seeks to investigate the utility of implementing video recording by scoring OSCEs in real-time using live examiners, and afterwards using video examiners from within and outside the learners' home institution. Using checklists, learners (n=33) were assessed by one live examiner and five video examiners on three OSCE stations: suturing, arthrocentesis, and thoracocentesis. When stations were considered collectively, there was no difference between pass/fail outcome between live and video examiners (χ2 = 0.37, p = .55). However, when considered individually, stations (χ2 = 16.64, p < .001) and interaction between station and type of examiner (χ2 = 7.13, p = .03) demonstrated a significant effect on pass/fail outcome. Specifically, learners being assessed on suturing with a video examiner had increased odds of passing the station as compared with their arthrocentesis or thoracocentesis stations. Internal consistency was fair to moderate (0.34-0.45). Inter-rater reliability measures varied but were mostly moderate to strong (0.56-0.82). Video examiners spent longer assessing learners than live raters (mean of 21 min/learner vs. 13 min/learner). Station-specific differences among video examiners may be due to intermittent visibility issues during video capture. Overall, video recording learner performances appears reliable and feasible, although there were time, cost, and technical issues that may limit its routine use.

Collaborative Development of Core Entrustable Professional Activities for Veterinary Education.

Entrustable professional activities (EPAs) have been proposed as a practical framework for the implementation of competency-based education. As veterinary education moves toward a competency-based approach, core EPAs provide a context for assessment of workplace activities. This article reports on the development of eight core clinical EPAs for veterinary education created through multi-institutional collaboration, with international input from veterinary educators and veterinary educational leaders. These core EPAs are intended as minimal expectations for clinical activities that should be assessed for every graduate of Association of American Veterinary Medical Colleges member institutions. Adoption of the core EPAs and the associated Competency-Based Veterinary Education (CBVE) framework by veterinary schools is intended to promote Day One graduate competence and thereby enhance patient care and client service.

Collaborative Development of a Shared Framework for Competency-Based Veterinary Education.

Competency-based medical education is an educational innovation implemented in health professions worldwide as a means to ensure graduates meet patient and societal needs. The focus on student-centered education and programmatic outcomes offers a series of benefits to learners, institutions and society. However, efforts to establish a shared, comprehensive competency-based framework in veterinary education have lagged. This article reports on the development and outcome of a competency-based veterinary education (CBVE) framework created through multi-institutional collaboration with international input from veterinary educators and veterinary educational leaders. The CBVE Framework is designed to reflect the competencies expected of new graduates from member institutions of the Association of American Veterinary Medical Colleges (AAVMC). The CBVE Framework consists of nine domains of competence and 32 competencies, each supplemented with illustrative sub-competencies to guide veterinary schools in implementing competency-based education in their local context. The nine domains of competence are: clinical reasoning and decision-making; individual animal care and management; animal population care and management; public health; communication; collaboration; professionalism and professional identity; financial and practice management; and scholarship. Developed through diverse input to facilitate broad adoption, the CBVE Framework provides the foundation for competency-based curricula and outcomes assessment in veterinary education internationally. We believe that other groups seeking to design a collective product for broad adoption might find useful the methods used to develop the CBVE Framework, including establishing expertise diversity within a small-to-medium size working group, soliciting progressive input and feedback from stakeholders, and engaging in consensus building and critical reflection throughout the development process.

Development of core entrustable professional activities linked to a competency-based veterinary education framework.

Purpose: Despite the adoption of competency-based education in some veterinary schools over the past 15 years, only recently has a concerted effort been directed toward this in veterinary education internationally.Methods: In 2015, educational leaders from the Association of American Veterinary Medical Colleges (AAVMC) member schools came together with a strong call to action to create shared tools for clinical competency assessment.Results: This resulted in the formation of the AAVMC Competency-Based Veterinary Education (CBVE) Working Group, which then embarked on the creation of a shared competency framework and the development of eight core entrustable professional activities (EPAs) linked to this framework.Conclusions: This paper will report on the development of these EPAs and their integration with the concurrently-developed CBVE Framework.

A Survey of Established Veterinary Clinical Skills Laboratories from Europe and North America: Present Practices and Recent Developments.

Developing competence in clinical skills is important if graduates are to provide entry-level care, but it is dependent on having had sufficient hands-on practice. Clinical skills laboratories provide opportunities for students to learn on simulators and models in a safe environment and to supplement training with animals. Interest in facilities for developing veterinary clinical skills has increased in recent years as many veterinary colleges face challenges in training their students with traditional methods alone. For the present study, we designed a survey to gather information from established veterinary clinical skills laboratories with the aim of assisting others considering opening or expanding their own facility. Data were collated from 16 veterinary colleges in North America and Europe about the uses of their laboratory, the building and associated facilities, and the staffing, budgets, equipment, and supporting learning resources. The findings indicated that having a dedicated veterinary clinical skills laboratory is a relatively new initiative and that colleges have adopted a range of approaches to implementing and running the laboratory, teaching, and assessments. Major strengths were the motivation and positive characteristics of the staff involved, providing open access and supporting self-directed learning. However, respondents widely recognized the increasing demands placed on the facility to provide more space, equipment, and staff. There is no doubt that veterinary clinical skills laboratories are on the increase and provide opportunities to enhance student learning, complement traditional training, and benefit animal welfare.

Evaluation of Veterinary Student Surgical Skills Preparation for Ovariohysterectomy Using Simulators: A Pilot Study.

This paper describes the development and evaluation of training intended to enhance students' performance on their first live-animal ovariohysterectomy (OVH). Cognitive task analysis informed a seven-page lab manual, 30-minute video, and 46-item OVH checklist (categorized into nine surgery components and three phases of surgery). We compared two spay simulator models (higher-fidelity silicone versus lower-fidelity cloth and foam). Third-year veterinary students were randomly assigned to a training intervention: lab manual and video only; lab manual, video, and $675 silicone-based model; lab manual, video, and $64 cloth and foam model. We then assessed transfer of training to a live-animal OVH. Chi-square analyses determined statistically significant differences between the interventions on four of nine surgery components, all three phases of surgery, and overall score. Odds ratio analyses indicated that training with a spay model improved the odds of attaining an excellent or good rating on 25 of 46 checklist items, six of nine surgery components, all three phases of surgery, and the overall score. Odds ratio analyses comparing the spay models indicated an advantage for the $675 silicon-based model on only 6 of 46 checklist items, three of nine surgery components, and one phase of surgery. Training with a spay model improved performance when compared to training with a manual and video only. Results suggested that training with a lower-fidelity/cost model might be as effective when compared to a higher-fidelity/cost model. Further research is required to investigate simulator fidelity and costs on transfer of training to the operational environment.

Using cognitive task analysis to create a teaching protocol for bovine dystocia.

When learning skilled techniques and procedures, students face many challenges. Learning is easier when detailed instructions are available, but experts often find it difficult to articulate all of the steps involved in a task or relate to the learner as a novice. This problem is further compounded when the technique is internal and unsighted (e.g., obstetrical procedures). Using expert bovine practitioners and a life-size model cow and calf, the steps and decision making involved in performing correction of two different dystocia presentations (anterior leg back and breech) were deconstructed using cognitive task analysis (CTA). Video cameras were positioned to capture movement inside and outside the cow model while the experts were asked to first perform the technique as they would in a real situation and then perform the procedure again as if articulating the steps to a novice learner. The audio segments were transcribed and, together with the video components, analyzed to create a list of steps for each expert. Consensus was achieved between experts during individual interviews followed by a group discussion. A "gold standard" list or teaching protocol was created for each malpresentation. CTA was useful in defining the technical and cognitive steps required to both perform and teach the tasks effectively. Differences between experts highlight the need for consensus before teaching the skill. In addition, the study identified several different, yet effective, techniques and provided information that could allow experts to consider other approaches they might use when their own technique fails.

Longitudinal assessment of competency development at The Ohio State University using the competency-based veterinary education (CBVE) model.

With the development of the American Association of Veterinary Medical Colleges' Competency-Based Veterinary Education (CBVE) model, veterinary schools are reorganizing curricula and assessment guidelines, especially within the clinical rotation training elements. Specifically, programs are utilizing both competencies and entrustable professional activities (EPAs) as opportunities for gathering information about student development within and across clinical rotations. However, what evidence exists that use of the central tenets of the CBVE model (competency framework, milestones and EPAs) improves our assessment practices and captures reliable and valid data to track competency development of students as they progress through their clinical year? Here, we report on validity evidence to support the use of scores from in-training evaluation report forms (ITERs) and workplace-based assessments of EPAs to evaluate competency progression within and across domains described in the CBVE, during the final year clinical training period of The Ohio State University's College of Veterinary Medicine (OSU-CVM) program. The ITER, used at the conclusion of each rotation, was modified to include the CBVE competencies that were assessed by identifying the stage of student development on a series of descriptive milestones (from pre-novice to competent). Workplace based assessments containing entrustment scales were used to assess EPAs from the CBVE model within each clinical rotation. Competency progression and entrustment scores were evaluated on each of the 31 rotations offered and high-stakes decisions regarding student performance were determined by a collective review of all the ITERs and EPAs recorded for each learner across each semester and the entire year. Results from the class of 2021, collected on approximately 190 students from 31 rotations, are reported with more than 55 299 total competency assessments combined with milestone placement and 2799 complete EPAs. Approximately 10% of the class was identified for remediation and received additional coaching support. Data collected longitudinally through the ITER on milestones provides initial validity evidence to support using the scores in higher stakes contexts such as identifying students for remediation and for determining whether students have met the necessary requirements to successfully complete the program. Data collected on entrustment scores did not, however, support such decision making. Implications are discussed.

Supporting diversity, equity, inclusion, and belonging to strengthen and position the veterinary profession for service, sustainability, excellence, and impact.

Advancing equality and equity in society is creating positive change, and the time has come to critically evaluate veterinary medicine, which, by all metrics, lacks diversity. To keep pace with increasingly diverse demographics and recent surges in pet ownership among all racial/ethnic groups, significant efforts to enhance diversity, equity, inclusion, and belonging (DEIB) must occur in veterinary colleges and the profession. Recruiting more underrepresented students, building pipelines for diverse faculty/staff, and creating inclusive, welcoming environments where all can thrive are critical steps toward enhancing DEIB within our organizations and profession. Our goal is to share experiences and lessons learned from our intentional commitment to strengthen DEIB, with the hope that our journey will be helpful to others. Increasing diversity in the veterinary profession will be facilitated through removing barriers, creating inclusive work environments where all people feel they belong, and ensuring fair and equitable hiring and personnel management practices. These steps should in turn improve access and quality of veterinary care, ensure we are more representative of the communities we serve, increase revenue, and preserve the human-animal bond. "You cannot change any society unless you take responsibility for it, unless you see yourself belonging to it, and responsible for changing it." - Grace Lee Boggs.

Assessment of first-year veterinary students' clinical skills using objective structured clinical examinations.

The DVM program at the University of Calgary offers a Clinical Skills course each year for the first three years. The course is designed to teach students the procedural skills required for entry-level general veterinary practice. Objective Structured Clinical Examinations (OSCEs) were used to assess students' performance on these procedural skills. A series of three OSCEs were developed for the first year. Content was determined by an exam blueprint, exam scoring sheets were created, rater training was provided, a mock OSCE was performed with faculty and staff, and the criterion-referencing Ebel method was used to set cut scores for each station using two content experts. Each station and the overall exam were graded as pass or fail. Thirty first-year DVM students were assessed. Content validity was ensured by the exam blueprint and expert review. Reliability (coefficient α) of the stations from the three OSCE exams ranged from 0.0 to 0.71. The three exam reliabilities (Generalizability Theory) were, for OSCE 1, G=0.56; OSCE 2, G=0.37; and OSCE 3, G=0.32. Preliminary analysis has suggested that the OSCEs demonstrate face and content validity, and certain stations demonstrated adequate reliability. Overall exam reliability was low, which reflects issues with first-time exam delivery. Because this year was the first that this course was taught and this exam format was used, work continues in the program on the teaching of the procedural skills and the development and revision of OSCE stations and scoring checklists.

The use of global rating scales for OSCEs in veterinary medicine.

OSCEs (Objective Structured Clinical Examinations) are widely used in health professions to assess clinical skills competence. Raters use standardized binary checklists (CL) or multi-dimensional global rating scales (GRS) to score candidates performing specific tasks. This study assessed the reliability of CL and GRS scores in the assessment of veterinary students, and is the first study to demonstrate the reliability of GRS within veterinary medical education. Twelve raters from two different schools (6 from University of Calgary [UCVM] and 6 from Royal (Dick) School of Veterinary Studies [R(D)SVS] were asked to score 12 students (6 from each school). All raters assessed all students (video recordings) during 4 OSCE stations (bovine haltering, gowning and gloving, equine bandaging and skin suturing). Raters scored students using a CL, followed by the GRS. Novice raters (6 R(D)SVS) were assessed independently of expert raters (6 UCVM). Generalizability theory (G theory), analysis of variance (ANOVA) and t-tests were used to determine the reliability of rater scores, assess any between school differences (by student, by rater), and determine if there were differences between CL and GRS scores. There was no significant difference in rater performance with use of the CL or the GRS. Scores from the CL were significantly higher than scores from the GRS. The reliability of checklist scores were .42 and .76 for novice and expert raters respectively. The reliability of the global rating scale scores were .7 and .86 for novice and expert raters respectively. A decision study (D-study) showed that once trained using CL, GRS could be utilized to reliably score clinical skills in veterinary medicine with both novice and experienced raters.

In situ coagulation and transection of the ovarian pedicle: an alternative to laparoscopic ovariectomy in juvenile horses.

The feasibility of leaving the ovaries within the peritoneal cavity after laparoscopic coagulation and transection of the ovarian pedicle was assessed in the juvenile horse. Elective ovariectomy was performed on 10 quarter horses, aged 4 to 5 mo, with the fillies in a Trendelenburg position. The mesovarium was isolated, and multiple coagulation and transection cycles were performed until all ovarian attachments had been severed. The ovaries were dropped within the abdomen, and hemostasis of the transected mesovarium was evaluated before closure. The mean surgical time was 33 min (range, 23 to 48 min). Ten weeks after surgery the fillies were humanely euthanized. At postmortem examination, the ovary location within the abdomen was noted. In 1 horse, there was an abdominal adhesion; viscera had been punctured during insufflation. Of the 20 ovaries, 4 were free-floating within the abdominal cavity; the other 16 were enveloped in the free portion of the greater omentum in the cranioventral abdomen. Histologic examination of the ovaries was performed to assess follicle cell viability. In both the free-floating and the attached ovaries, the deep blood vessels and all examined follicular structures were necrotic and partially mineralized. Laparoscopic electrosurgical transection of the ovarian pedicle without removal of the ovaries should be considered an alternative to other ovariectomy techniques that may be performed in young female horses.

Cardiopulmonary effects and induction and recovery characteristics of isoflurane and sevoflurane in foals.

OBJECTIVE: To compare induction and recovery characteristics and cardiopulmonary effects of isoflurane and sevoflurane in foals. DESIGN: Prospective crossover study. ANIMALS: 6 healthy foals. PROCEDURE: Foals were anesthetized twice (once at 1 month of age and again at 3 months of age). Anesthesia was induced by administration of the agent in oxygen through a nasotracheal tube. During maintenance of anesthesia, foals were positioned in dorsal recumbency; intermittent positive-pressure ventilation was performed. Characteristics of induction and recovery were recorded. Cardiopulmonary variables were recorded 10 minutes after anesthetic induction and 15, 30, 45, and 60 minutes later. RESULTS: All 6 foals were successfully anesthetized with isoflurane and sevoflurane. There were no significant differences between the 2 drugs in regard to characteristics of induction or recovery, and induction and recovery were generally smooth and unremarkable. There were no significant differences between drugs in regard to measured cardiopulmonary variables; however, both drugs caused initial hypotension that resolved over time. CONCLUSIONS AND CLINICAL RELEVANCE: Results suggest that isoflurane and sevoflurane can both be used for general anesthesia of 1- to 3-month-old foals. Significant differences between the 2 agents were not detected for any of the variables measured, suggesting that quality of anesthesia with these 2 agents was comparable.

Effect of hemi-circumferential periosteal transection and elevation in foals with experimentally induced angular limb deformities.

OBJECTIVE: To evaluate the effect of hemi-circumferential periosteal transection and elevation (HCPTE) in foals with, experimentally induced angular limb deformities. DESIGN: Prospective study. ANIMALS: 10 healthy foals. PROCEDURE: When foals were 30 days old, transphyseal bridge implants were placed on the lateral aspects of both distal radial physes. At 90 days of age (or when 15 degrees of angulation had developed), implants were removed, and HCPTE was performed on 1 limb. Foals were confined in small pens after surgery; the front feet of the foals were rasped weekly to maintain medial-to-lateral hoof wall balance. Dorsopalmar radiographic projections of the carpi were obtained before HCPTE and 2, 4, 6, 8, and 48 weeks later. RESULTS: At the time of transphyseal bridge removal and HCPTE, both treated and control limbs were observed to have a significantly greater carpal valgus, compared with the initial degree of angulation at 30 days of age. Following HCPTE or sham surgery, all limbs straightened over the subsequent 2 months of the study. Median angulation was not significantly different between treated and control limbs at any time during the study. CONCLUSIONS AND CLINICAL RELEVANCE: Results suggest that in foals with experimentally induced limb deformities, HCPTE was no more effective than stall confinement and hoof trimming alone for correction of the deformity.

Clinical observations surrounding an increased incidence of postanesthetic myopathy in halothane-anesthetized horses.

BACKGROUND: During an 8-year period of clinic improvements, an increased incidence of postanesthetic myopathy (PAM) was observed in horses undergoing anesthesia. A request was made for an independent anesthesiologist to examine possible reasons for this increase. Routine methods used for anesthesia were then compared with new methods introduced by the anesthesiologist. OBSERVATIONS: Investigative observations were conducted from October 1999 to April 2000. Following premedication with xylazine or detomidine, anesthesia was induced with diazepam or guiaifenesin followed by ketamine. Maintenance of anesthesia was with halothane/O(2) with mechanical ventilation. Information was gathered from observing routine anesthetics. Seventeen horses were designated group A (October-December 1999) for which hypotension was defined as mean arterial pressure (MAP) <60 mmHg and managed using low-volume crystalloid fluid therapy, etilefrine, or dopamine. In the next 18 horses (group B), hypotension was redefined as MAP <70 mmHg, and was managed using high-volume fluid therapy and dobutamine (January-April 2000). No other changes were made. In both groups, creatine kinase (CK) and aspartate transaminase (AST) concentrations were measured before induction, 1 and 25 hours after standing. Occurrence of PAM was defined as CK and/or AST concentrations above 1000 IU L(-1) and analyzed with Fisher's exact test between groups. Muscle enzymes increased in five horses in group A (two with clinical signs) and in one horse in group B (difference between groups: p = 0.088). CONCLUSIONS: Maintaining MAP >70 mmHg with high-volume fluid therapy and dobutamine may help to reduce the severity of myopathy.