Pathologists aren't pigeons: exploring the neural basis of visual recognition and perceptual expertise in pathology.

Visual (perceptual) reasoning is a critical skill in many medical specialties, including pathology, diagnostic imaging, and dermatology. However, in an ever-compressed medical curriculum, learning and practicing this skill can be challenging. Previous studies (including work with pigeons) have suggested that using reward-feedback-based activities, novices can gain expert levels of visual diagnostic accuracy in shortened training times. But is this level of diagnostic accuracy a result of image recognition (categorization) or is it the acquisition of diagnostic expertise? To answer this, the authors measured electroencephalographic data (EEG) and two components of the human event-related brain potential (reward positivity and N170) to explore the nature of visual expertise in a novice-expert study in pathology visual diagnosis. It was found that the amplitude of the reward positivity decreased with learning in novices (suggesting a decrease in reliance on feedback, as in other studies). However, this signal remained significantly different from the experts whose reward positivity signal did not change over the course of the experiment. There were no changes in the amplitude of the N170 (a reported neural marker of visual expertise) in novices over time. Novice N170 signals remained statistically and significantly lower in amplitude compared to experts throughout task performance. These data suggest that, while novices gained the ability to recognize (categorize) pathologies through reinforcement learning as quantified by the change in reward positivity, increased accuracy, and decreased time for responses, there was little change in the neural marker associated with visual expertise (N170). This is consistent with the multi-dimensional and complex nature of visual expertise and provides insight into future training programs for novices to bridge the expertise gap.

Exploring the development of interprofessional competence and professional identity: A Situated Learning Theory study.

Workplace-based learning provides medical students exposure to interprofessional competencies through repeated exposures and active participation in interprofessional learning activities. Using Situated Learning Theory as our theoretical lens, we explored with medical students how interacting with existing interprofessional teams contributes to development of an expanded health care professional identity. An embedded mixed methods study using semi-structured interviews and questionnaires to assess readiness for interprofessional learning was conducted with 14 medical students completing an elective at an interprofessional pain medicine clinic. Within this workplace-based context, a model identifying key themes and supporting factors contributing to the development of an extended professional identity was developed. These findings help describe the processes by which students gain interprofessional collaboration competence.

What role does basic research have in an applied field?

Research in health professions education has often been portrayed as an applied field, one that draws on more basic forms of research in pursuing what are primarily practical ends. While there is an undeniable practical side to much of the work published in our field, and in this Journal in particular, this can be problematic when the necessary basic research is not extant. In this editorial, two of this Journal's editors consider some of the challenges in bridging these basic research gaps in an erstwhile applied field, and the implications for the kinds of research we undertake and for the identity of the field as a whole.

Building Clinical Skills Teaching Materials: A Study of Cognitive Task Analysis Versus Free Recall Instruction to Teach Canine Indirect Ophthalmoscopy.

Ophthalmoscopy is a core component of a complete ophthalmic examination. Due to its complex technical aspects and patients becoming uncooperative if the procedure is prolonged, it can be a difficult skill for a novice to learn and develop proficiency in. Skills instruction is typically provided by subject matter experts (SMEs) through free recall without an agreed-upon instructional framework. This can lead to unintentional omission of essential steps and knowledge required to perform skills correctly. Cognitive task analysis (CTA) allows for construction of standardized instructional protocols that encompass the knowledge and skills experts apply when performing tasks. The objectives of this study were to (a) develop a CTA-based teaching protocol for canine indirect ophthalmoscopy and (b) compare the steps verbalized or demonstrated by SMEs during free recall instruction versus those ultimately identified by CTA. Four SMEs participated in free recall instructional sessions and interviews used for the development of a CTA-based teaching protocol for novice learners. The CTA-based protocol identified 66 steps and sub-steps considered essential for successfully performing canine indirect ophthalmoscopy. During instructional sessions, SMEs on average failed to verbalize 57.1% of clinical knowledge steps, did not verbalize or demonstrate 68.3% and 9.5% of action steps, and did not verbalize or demonstrate 73.2% and 40.4% of decision steps, respectively. This study demonstrates that SMEs teaching indirect ophthalmoscopy by free recall may unintentionally omit important steps, suggesting that compared with free recall, CTA may generate more comprehensive and thus potentially more effective instructional materials for teaching technical skills in veterinary medicine.

What Matters Most to Residency and Intern Selection Committees in Veterinary Medicine?

A recent survey of members of residency selection committees for the American College of Veterinary Internal Medicine and American College of Veterinary Surgeons boards found letters of recommendation to be the most important factor when reviewing a resident's application followed by class rank as the second most important factor. These statistics indicate an interesting, but possibly troubling trend. This Letter to The Editor discusses the major problems concerning these findings and what residency program committees might consider as an alternative.

Comparing Entrustment and Competence: An Exploratory Look at Performance-Relevant Information in the Final Year of a Veterinary Program.

Workplace-based assessments and entrustment scales have two primary goals: providing formative information to assist students with future learning; and, determining if and when learners are ready for safe, independent practice. To date, there has not been an evaluation of the relationship between these performance-relevant information pieces in veterinary medicine. This study collected quantitative and qualitative data from a single cohort of final-year students (n = 27) across in-training evaluation reports (ITERs) and entrustment scales in a distributed veterinary hospital environment. Here we compare progression in scoring and performance within and across student, within and across method of assessment, over time. Narrative comments were quantified using the Completed Clinical Evaluation Report Rating (CCERR) instrument to assess quality of written comments. Preliminary evidence suggests that we may be capturing different aspects of performance using these two different methods. Specifically, entrustment scale scores significantly increased over time, while ITER scores did not. Typically, comments on entrustment scale scores were more learner specific, longer, and used more of a coaching voice. Longitudinal evaluation of learner performance is important for learning and demonstration of competence; however, the method of data collection could influence how feedback is structured and how performance is ultimately judged.

Development of and Validity Evidence for a Canine Ocular Model for Training Novice Veterinary Students to Perform a Fundic Examination.

Indirect fundoscopy is challenging for novice learners, as patients are often intolerant of the procedure, impeding development of proficiency. To address this, we developed a canine ocular simulator that we hypothesized would improve student learning compared to live dogs. Six board-certified veterinary ophthalmologists and 19 second-year veterinary students (novices) performed an indirect fundic examination on the model and live dog. Prior to assessment, novices were introduced to the skill with a standardized teaching protocol and practiced (without feedback) with either the model (n = 10) or live dog (n = 9) for 30 minutes. All participants evaluated realism and usefulness of the model using a Likert-type scale. Performance on the live dog and model was evaluated in all participants using time to completion of task, performance of fundic examination using a checklist and global score, identification of objects in the fundus of the model, and evaluation of time spent looking at the fundus of the model using eye tracking. Novices (trained on simulator or live dogs) were compared in fundic examination performance on the live dog and identification of shapes in the model. In general, experts performed the fundic examination faster (p ≤ .0003) and more proficiently than the novices, although there were no differences in eye tracking behavior between groups (p ≥ .06). No differences were detected between training on simulator versus live dog in development of fundoscopy skills in novices (p ≥ .20). These findings suggest that this canine model may be an effective tool to train students to perform fundoscopy.

Selection for family medicine residency training in Canada: How consistently are the same students ranked by different programs?

OBJECTIVE: To examine the consistency of the ranking of Canadian and US medical graduates who applied to Canadian family medicine (FM) residency programs between 2007 and 2013. DESIGN: Descriptive cross-sectional study. SETTING: Family medicine residency programs in Canada. PARTICIPANTS: All 17 Canadian medical schools allowed access to their anonymized program rank-order lists of students applying to FM residency programs submitted to the first iteration of the Canadian Resident Matching Service match from 2007 to 2013. MAIN OUTCOME MEASURES: The rank position of medical students who applied to more than 1 FM residency program on the rank-order lists submitted by the programs. Anonymized ranking data submitted to the Canadian Resident Matching Service from 2007 to 2013 by all 17 FM residency programs were used. Ranking data of eligible Canadian and US medical graduates were analyzed to assess the within-student and between-student variability in rank score. These covariance parameters were then used to calculate the intraclass correlation coefficient (ICC) for all programs. Program descriptions and selection criteria were also reviewed to identify sites with similar profiles for subset ICC analysis. RESULTS: Between 2007 and 2013, the consistency of ranking by all programs was fair at best (ICC = 0.34 to 0.39). The consistency of ranking by larger urban-based sites was weak to fair (ICC = 0.23 to 0.36), and the consistency of ranking by sites focusing on training for rural practice was weak to moderate (ICC = 0.16 to 0.55). CONCLUSION: In most cases, there is a low level of consistency of ranking of students applying for FM training in Canada. This raises concerns regarding fairness, particularly in relation to expectations around equity and distributive justice in selection processes.

Sedation levels in dogs: a validation study.

BACKGROUND: The aim of this study was to assess validation evidence for a sedation scale for dogs. We hypothesized that the chosen sedation scale would be unreliable when used by different raters and show poor discrimination between sedation protocols. A sedation scale (range 0-21) was used to score 62 dogs scheduled to receive sedation at two veterinary clinics in a prospective trial. Scores recorded by a single observer were used to assess internal consistency and construct validity of the scores. To assess inter-rater reliability, video-recordings of sedation assessment were randomized and blinded for viewing by 5 raters untrained in the scale. Videos were also edited to allow assessment of inter-rater reliability of an abbreviated scale (range 0-12) by 5 different raters. RESULTS: Both sedation scales exhibited excellent internal consistency and very good inter-rater reliability (full scale, intraclass correlation coefficient [ICCsingle] = 0.95; abbreviated scale, ICCsingle = 0.94). The full scale discriminated between the most common protocols: dexmedetomidine-hydromorphone (median [range] of sedation score, 11 [1-18], n = 20) and acepromazine-hydromorphone (5 [0-15], n = 36, p = 0.02). CONCLUSIONS: The hypothesis was rejected. Full and abbreviated scales showed excellent internal consistency and very good reliability between multiple untrained raters. The full scale differentiated between levels of sedation.

Hemispheric activation differences in novice and expert clinicians during clinical decision making.

Clinical decision making requires knowledge, experience and analytical/non-analytical types of decision processes. As clinicians progress from novice to expert, research indicates decision-making becomes less reliant on foundational biomedical knowledge and more on previous experience. In this study, we investigated how knowledge and experience were reflected in terms of differences in neural areas of activation. Novice and expert clinicians diagnosed simple or complex (easy, hard) cases while functional magnetic resonance imaging (fMRI) data were collected. Our results highlight key differences in the neural areas activated in novices and experts during the clinical decision-making process. fMRI data were collected from ten second year medical students (novices) and ten practicing gastroenterologists (experts) while they diagnosed sixteen (eight easy and eight hard) clinical cases via multiple-choice questions. Behavioral data were collected for diagnostic accuracy (correct/incorrect diagnosis) and time taken to assign a clinical diagnosis. Two analyses were performed with the fMRI data. First, data from easy and hard cases were compared within respective groups (easy > hard, hard > easy). Second, neural differences between novices and experts (novice > expert, expert > novice) were assessed. Experts correctly diagnosed more cases than novices and made their diagnoses faster than novices on both easy and hard cases (all p's < 0.05). Time taken to diagnose hard cases took significantly longer for both novices and experts. While similar neural areas were activated in both novices and experts during the decision making process, we identified significant hemispheric activation differences between novice and expert clinicians when diagnosing hard clinical cases. Specifically, novice clinicians had greater activations in the left anterior temporal cortex and left ventral lateral prefrontal cortex whereas expert clinicians had greater activations in the right dorsal lateral, right ventral lateral, and right parietal cortex. Hemispheric differences in activation were not observed between novices and experts while diagnosing easy clinical cases. While clinical decision-making engaged the prefrontal cortex (PFC) in both novices and experts, interestingly we observed expertise related differences in the regions and hemispheres of PFC activation between these groups for hard clinical cases. Specifically, in novices we observed activations in left hemisphere neural regions associated with factual rule-based knowledge, whereas in experts we observed right hemisphere activation in neural regions associated with experiential knowledge. Importantly, at the neural level, our data highlight differences in so called type 2 clinical decision-making processes related to prior knowledge and experience.

Working memory, reasoning, and expertise in medicine-insights into their relationship using functional neuroimaging.

Clinical reasoning is dependent upon working memory (WM). More precisely, during the clinical reasoning process stored information within long-term memory is brought into WM to facilitate the internal deliberation that affords a clinician the ability to reason through a case. In the present study, we examined the relationship between clinical reasoning and WM while participants read clinical cases with functional magnetic resonance imaging (fMRI). More specifically, we examined the impact of clinical case difficulty (easy, hard) and clinician level of expertise (2nd year medical students, senior gastroenterologists) on neural activity within regions of cortex associated with WM (i.e., the prefrontal cortex) during the reasoning process. fMRI was used to scan ten second-year medical students and ten practicing gastroenterologists while they reasoned through sixteen clinical cases [eight straight forward (easy) and eight complex (hard)] during a single 1-h scanning session. Within-group analyses contrasted the easy and hard cases which were then subsequently utilized for a between-group analysis to examine effects of expertise (novice > expert, expert > novice). Reading clinical cases evoked multiple neural activations in occipital, prefrontal, parietal, and temporal cortical regions in both groups. Importantly, increased activation in the prefrontal cortex in novices for both easy and hard clinical cases suggests novices utilize WM more so than experts during clinical reasoning. We found that clinician level of expertise elicited differential activation of regions of the human prefrontal cortex associated with WM during clinical reasoning. This suggests there is an important relationship between clinical reasoning and human WM. As such, we suggest future models of clinical reasoning take into account that the use of WM is not consistent throughout all clinical reasoning tasks, and that memory structure may be utilized differently based on level of expertise.

Understanding Reliability: A Review for Veterinary Educators.

Veterinary medical faculty and administrators routinely administer student assessments and conduct surveys to make decisions regarding student performance and to assess their courses/curricula. The decisions that are made are a result of the scores generated. However, how reliable are the scores and how confident can we be about these decisions? Reliability is one of the hallmarks of validity evidence, but what does this mean and what affects the reliability of scores? The purpose of this article is to provide veterinary medical educators and administrators with fundamental information regarding the concept of reliability. Specifically, we review what sources of error reduce the reliability of scores and we describe the different types of reliability coefficients that are reported.

Rater Errors in Clinical Performance Assessments.

Rater errors are some of the most significant validity threats to any performance assessment. Veterinary medical education routinely uses raters to assess student performance in a variety of scenarios (e.g., clinical assessments, OSCEs, etc.). The purpose of this "teaching tip" is to introduce veterinary medical educators to the notion of rater error, identify a list of common rater errors, and discuss how these errors can be addressed and minimized so as to produce accurate and defensible measures of student performance.

Exploring the Teaching Motivations, Satisfaction, and Challenges of Veterinary Preceptors: A Qualitative Study.

Optimization of clinical veterinary education requires an understanding of what compels veterinary preceptors in their role as clinical educators, what satisfaction they receive from the teaching experience, and what struggles they encounter while supervising students in private practice. We explored veterinary preceptors' teaching motivations, enjoyment, and challenges by undertaking a thematic content analysis of 97 questionnaires and 17 semi-structured telephone interviews. Preceptor motivations included intrinsic factors (obligation to the profession, maintenance of competence, satisfaction) and extrinsic factors (promotion of the veterinary field, recruitment). Veterinarians enjoyed observing the learner (motivation and enthusiasm, skill development) and engaging with the learner (sharing their passion for the profession, developing professional relationships). Challenges for veterinary preceptors included variability in learner interest and engagement, time management, and lack of guidance from the veterinary medicine program. We found dynamic interactions among the teaching motivations, enjoyment, and challenges for preceptors. Our findings suggest that in order to sustain the veterinary preceptor, there is a need to recognize the interplay between the incentives and disincentives for teaching, to foster the motivations and enjoyment for teaching, and to mitigate the challenges of teaching in community private practice.

Supporting Veterinary Preceptors in a Distributed Model of Education: A Faculty Development Needs Assessment.

Effective faculty development for veterinary preceptors requires knowledge about their learning needs and delivery preferences. Veterinary preceptors at community practice locations in Alberta, Canada, were surveyed to determine their confidence in teaching ability and interest in nine faculty development topics. The study included 101 veterinarians (48.5% female). Of these, 43 (42.6%) practiced veterinary medicine in a rural location and 54 (53.5%) worked in mixed-animal or food-animal practice. Participants reported they were more likely to attend an in-person faculty development event than to participate in an online presentation. The likelihood of attending an in-person event differed with the demographics of the respondent. Teaching clinical reasoning, assessing student performance, engaging and motivating students, and providing constructive feedback were topics in which preceptors had great interest and high confidence. Preceptors were least confident in the areas of student learning styles, balancing clinical workload with teaching, and resolving conflict involving the student. Disparities between preceptors' interest and confidence in faculty development topics exist, in that topics with the lowest confidence scores were not rated as those of greatest interest. While the content and format of clinical teaching faculty development events should be informed by the interests of preceptors, consideration of preceptors' confidence in teaching ability may be warranted when developing a faculty development curriculum.

Final-Year Students' and Clinical instructors' Experience of Workplace-Based Assessments Used in a Small-Animal Primary-Veterinary-Care Clinical Rotation.

Final-year veterinary students must meet baseline clinical competency upon completion of their training for entry to practice. Workplace-based assessments (WBAs), widely used in human medical training to assess post-graduate students' professionalism and clinical performance, have recently been adopted in undergraduate veterinary clinical teaching environments. WBAs should support veterinary trainees' learning in a clinical teaching environment, though utility of WBAs within veterinary education may differ from that in medical training due to differences in context and in learners' stage of clinical development. We conducted focus groups with final-year veterinary students and clinical instructors following the implementation of three WBAs (Direct Observation of Procedural Skills [DOPS], the Mini-Clinical evaluation exercise [Mini-CEX], and the In-Training Evaluation Report [ITER]) during a small-animal primary-veterinary-care rotation. Students and clinical instructors viewed the DOPS and Mini-CEX as feasible and valuable learning and assessment tools that offered an overall opportunity for timely in-the-moment feedback. Instructors viewed the ITER as less feasible in the context of a service-oriented veterinary clinical teaching environment. Students believed the ITER had potential to be informative, although in its existing application the ITER had limited utility due to time constraints on instructors that prevented them from providing students with individualized and specific feedback. In service-oriented veterinary clinical teaching environments, successful implementation of WBAs requires balancing provision of feedback to students, time demands on clinical instructors, and flexibility of assessment tools.

Changes in Veterinary Students' Attitudes Toward the Rural Environment and Rural Veterinary Practice: A Longitudinal Cohort Study.

There is a paucity of research regarding veterinary students' attitudes toward the rural environment and rural veterinary practice and how these attitudes might change over the course of a veterinary medicine program that includes rural clinical experience. Using a 23-item questionnaire, attitudes toward rural lifestyle, rural work-life balance, opportunities for career and skill development in rural veterinary practice, and inter-professional teamwork in the rural environment were assessed at the beginning and completion of a four-year veterinary medicine program. Eighty-six students (74.4% female) were included in this Canadian study over a six-year period. Thirty-one participants (36.1%) were rural students. Overall, students' attitudes toward the rural lifestyle, rural work-life balance, and inter-professional teamwork in rural veterinary practice all significantly decreased (p<.001) over the course of the program. As compared to urban students, rural students had significantly higher rural lifestyle scores at both the beginning (p<.001) and end (p<.01) of the veterinary medicine program. A less positive attitude toward living and working in a rural environment could influence students to exclude rural veterinary practice as a career choice. Rural clinical experiences designed to sustain or increase veterinary student interest in rural practice may not be sufficient to support positive rural attitudes. Given the demand for rural veterinary services in developed countries, the implications of this study may extend beyond Canada.

Does a Rater's Professional Background Influence Communication Skills Assessment?

There is increasing pressure in veterinary education to teach and assess communication skills, with the Objective Structured Clinical Examination (OSCE) being the most common assessment method. Previous research reveals that raters are a large source of variance in OSCEs. This study focused on examining the effect of raters' professional background as a source of variance when assessing students' communication skills. Twenty-three raters were categorized according to their professional background: clinical sciences (n=11), basic sciences (n=4), clinical communication (n=5), or hospital administrator/clinical skills technicians (n=3). Raters from each professional background were assigned to the same station and assessed the same students during two four-station OSCEs. Students were in year 2 of their pre-clinical program. Repeated-measures ANOVA results showed that OSCE scores awarded by the rater groups differed significantly: (F(matched_station_1) [2,91]=6.97, p=.002), (F(matched_station_2) [3,90]=13.95, p=.001), (F(matched_station_3) [3,90]=8.76, p=.001), and ((Fmatched_station_4) [2,91]=30.60, p=.001). A significant time effect between the two OSCEs was calculated for matched stations 1, 2, and 4, indicating improved student performances. Raters with a clinical communication skills background assigned scores that were significantly lower compared to the other rater groups. Analysis of written feedback provided by the clinical sciences raters showed that they were influenced by the students' clinical knowledge of the case and that they did not rely solely on the communication checklist items. This study shows that it is important to consider rater background both in recruitment and training programs for communication skills' assessment.

Measuring the effectiveness of small-group and web-based training methods in teaching clinical communication: a case comparison study.

Current teaching approaches in human and veterinary medicine across North America, Europe, and Australia include lectures, group discussions, feedback, role-play, and web-based training. Increasing class sizes, changing learning preferences, and economic and logistical challenges are influencing the design and delivery of communication skills in veterinary undergraduate education. The study's objectives were to (1) assess the effectiveness of small-group and web-based methods for teaching communication skills and (2) identify which training method is more effective in helping students to develop communication skills. At the Ross University School of Veterinary Medicine (RUSVM), 96 students were randomly assigned to one of three groups (control, web, or small-group training) in a pre-intervention and post-intervention group design. An Objective Structured Clinical Examination (OSCE) was used to measure communication competence within and across the intervention and control groups. Reliability of the OSCEs was determined by generalizability theory to be 0.65 (pre-intervention OSCE) and 0.70 (post-intervention OSCE). Study results showed that (1) small-group training was the most effective teaching approach in enhancing communication skills and resulted in students scoring significantly higher on the post-intervention OSCE compared to the web-based and control groups, (2) web-based training resulted in significant though considerably smaller improvement in skills than small-group training, and (3) the control group demonstrated the lowest mean difference between the pre-intervention/post-intervention OSCE scores, reinforcing the need to teach communication skills. Furthermore, small-group training had a significant effect in improving skills derived from the initial phase of the consultation and skills related to giving information and planning.

Using Machine Learning in Veterinary Medical Education: An Introduction for Veterinary Medicine Educators.

Machine learning (ML) offers potential opportunities to enhance the learning, teaching, and assessments within veterinary medical education including but not limited to assisting with admissions processes as well as student progress evaluations. The purpose of this primer is to assist veterinary educators in appraising and potentially adopting these rapid upcoming advances in data science and technology. In the first section, we introduce ML concepts and highlight similarities/differences between ML and classical statistics. In the second section, we provide a step-by-step worked example using simulated veterinary student data to answer a hypothesis-driven question. Python syntax with explanations is provided within the text to create a random forest ML prediction model, a model composed of decision trees with each decision tree being composed of nodes and leaves. Within each step of the model creation, specific considerations such as how to manage incomplete student records are highlighted when applying ML algorithms within the veterinary education field. The results from the simulated data demonstrate how decisions by the veterinary educator during ML model creation may impact the most important features contributing to the model. These results highlight the need for the veterinary educator to be fully transparent during the creation of ML models and future research is needed to establish guidelines for handling data not missing at random in medical education, and preferred methods for model evaluation.