Managing common ambulatory conditions: Exploring clinical decision making performance between pharmacists and family physicians.

Background: Prescribing is part of the expanded scope of practice for pharmacists in Alberta, Canada. Given these responsibilities, clinical decision making (the outcome from the diagnostic and therapeutic decision making process) is an essential skill for pharmacists. The current study compared diagnostic and therapeutic decision-making between Additional Prescribing Authority (APA) pharmacists and family physicians using a set of common ambulatory clinical cases that both practitioners could encounter in the community as part of their daily practice. Objectives: To explore clinical decision making performance and behaviors between APA pharmacists and family physicians during the assessment and prescribing of common ambulatory conditions. Methods: Eight written ambulatory clinical cases were developed by a panel of experts in both family medicine and pharmacy that were commonly encountered in both professions' daily practice. Participating APA pharmacists and family physicians reviewed the cases and responded with likely diagnoses, recommended treatments, and reported confidence in therapeutic choices. The responses of 18 APA pharmacists and 9 family physicians in community practices were analyzed. Results: There were no significant differences in diagnostic accuracy, therapeutic accuracy, confidence in diagnostic choices, and confidence in therapeutic choices between APA pharmacists and family physicians to these common ambulatory presentations. Conclusions: This study provides preliminary insights regarding the capabilities of pharmacists in the assessment of common ambulatory community conditions and suggests that APA pharmacists are making similar diagnostic and therapeutic decisions to family physicians. Future research could focus on examining the performance of pharmacists trained in different pharmacy education models, as well as their ability to provide clinical assessment in other specialties, or in more uncommon clinical scenarios.

Using Machine Learning in Veterinary Medical Education: An Introduction for Veterinary Medicine Educators.

Machine learning (ML) offers potential opportunities to enhance the learning, teaching, and assessments within veterinary medical education including but not limited to assisting with admissions processes as well as student progress evaluations. The purpose of this primer is to assist veterinary educators in appraising and potentially adopting these rapid upcoming advances in data science and technology. In the first section, we introduce ML concepts and highlight similarities/differences between ML and classical statistics. In the second section, we provide a step-by-step worked example using simulated veterinary student data to answer a hypothesis-driven question. Python syntax with explanations is provided within the text to create a random forest ML prediction model, a model composed of decision trees with each decision tree being composed of nodes and leaves. Within each step of the model creation, specific considerations such as how to manage incomplete student records are highlighted when applying ML algorithms within the veterinary education field. The results from the simulated data demonstrate how decisions by the veterinary educator during ML model creation may impact the most important features contributing to the model. These results highlight the need for the veterinary educator to be fully transparent during the creation of ML models and future research is needed to establish guidelines for handling data not missing at random in medical education, and preferred methods for model evaluation.

Pathologists aren't pigeons: exploring the neural basis of visual recognition and perceptual expertise in pathology.

Visual (perceptual) reasoning is a critical skill in many medical specialties, including pathology, diagnostic imaging, and dermatology. However, in an ever-compressed medical curriculum, learning and practicing this skill can be challenging. Previous studies (including work with pigeons) have suggested that using reward-feedback-based activities, novices can gain expert levels of visual diagnostic accuracy in shortened training times. But is this level of diagnostic accuracy a result of image recognition (categorization) or is it the acquisition of diagnostic expertise? To answer this, the authors measured electroencephalographic data (EEG) and two components of the human event-related brain potential (reward positivity and N170) to explore the nature of visual expertise in a novice-expert study in pathology visual diagnosis. It was found that the amplitude of the reward positivity decreased with learning in novices (suggesting a decrease in reliance on feedback, as in other studies). However, this signal remained significantly different from the experts whose reward positivity signal did not change over the course of the experiment. There were no changes in the amplitude of the N170 (a reported neural marker of visual expertise) in novices over time. Novice N170 signals remained statistically and significantly lower in amplitude compared to experts throughout task performance. These data suggest that, while novices gained the ability to recognize (categorize) pathologies through reinforcement learning as quantified by the change in reward positivity, increased accuracy, and decreased time for responses, there was little change in the neural marker associated with visual expertise (N170). This is consistent with the multi-dimensional and complex nature of visual expertise and provides insight into future training programs for novices to bridge the expertise gap.

Exploring the development of interprofessional competence and professional identity: A Situated Learning Theory study.

Workplace-based learning provides medical students exposure to interprofessional competencies through repeated exposures and active participation in interprofessional learning activities. Using Situated Learning Theory as our theoretical lens, we explored with medical students how interacting with existing interprofessional teams contributes to development of an expanded health care professional identity. An embedded mixed methods study using semi-structured interviews and questionnaires to assess readiness for interprofessional learning was conducted with 14 medical students completing an elective at an interprofessional pain medicine clinic. Within this workplace-based context, a model identifying key themes and supporting factors contributing to the development of an extended professional identity was developed. These findings help describe the processes by which students gain interprofessional collaboration competence.

Longitudinal assessment of competency development at The Ohio State University using the competency-based veterinary education (CBVE) model.

With the development of the American Association of Veterinary Medical Colleges' Competency-Based Veterinary Education (CBVE) model, veterinary schools are reorganizing curricula and assessment guidelines, especially within the clinical rotation training elements. Specifically, programs are utilizing both competencies and entrustable professional activities (EPAs) as opportunities for gathering information about student development within and across clinical rotations. However, what evidence exists that use of the central tenets of the CBVE model (competency framework, milestones and EPAs) improves our assessment practices and captures reliable and valid data to track competency development of students as they progress through their clinical year? Here, we report on validity evidence to support the use of scores from in-training evaluation report forms (ITERs) and workplace-based assessments of EPAs to evaluate competency progression within and across domains described in the CBVE, during the final year clinical training period of The Ohio State University's College of Veterinary Medicine (OSU-CVM) program. The ITER, used at the conclusion of each rotation, was modified to include the CBVE competencies that were assessed by identifying the stage of student development on a series of descriptive milestones (from pre-novice to competent). Workplace based assessments containing entrustment scales were used to assess EPAs from the CBVE model within each clinical rotation. Competency progression and entrustment scores were evaluated on each of the 31 rotations offered and high-stakes decisions regarding student performance were determined by a collective review of all the ITERs and EPAs recorded for each learner across each semester and the entire year. Results from the class of 2021, collected on approximately 190 students from 31 rotations, are reported with more than 55 299 total competency assessments combined with milestone placement and 2799 complete EPAs. Approximately 10% of the class was identified for remediation and received additional coaching support. Data collected longitudinally through the ITER on milestones provides initial validity evidence to support using the scores in higher stakes contexts such as identifying students for remediation and for determining whether students have met the necessary requirements to successfully complete the program. Data collected on entrustment scores did not, however, support such decision making. Implications are discussed.

What role does basic research have in an applied field?

Research in health professions education has often been portrayed as an applied field, one that draws on more basic forms of research in pursuing what are primarily practical ends. While there is an undeniable practical side to much of the work published in our field, and in this Journal in particular, this can be problematic when the necessary basic research is not extant. In this editorial, two of this Journal's editors consider some of the challenges in bridging these basic research gaps in an erstwhile applied field, and the implications for the kinds of research we undertake and for the identity of the field as a whole.

Building Clinical Skills Teaching Materials: A Study of Cognitive Task Analysis Versus Free Recall Instruction to Teach Canine Indirect Ophthalmoscopy.

Ophthalmoscopy is a core component of a complete ophthalmic examination. Due to its complex technical aspects and patients becoming uncooperative if the procedure is prolonged, it can be a difficult skill for a novice to learn and develop proficiency in. Skills instruction is typically provided by subject matter experts (SMEs) through free recall without an agreed-upon instructional framework. This can lead to unintentional omission of essential steps and knowledge required to perform skills correctly. Cognitive task analysis (CTA) allows for construction of standardized instructional protocols that encompass the knowledge and skills experts apply when performing tasks. The objectives of this study were to (a) develop a CTA-based teaching protocol for canine indirect ophthalmoscopy and (b) compare the steps verbalized or demonstrated by SMEs during free recall instruction versus those ultimately identified by CTA. Four SMEs participated in free recall instructional sessions and interviews used for the development of a CTA-based teaching protocol for novice learners. The CTA-based protocol identified 66 steps and sub-steps considered essential for successfully performing canine indirect ophthalmoscopy. During instructional sessions, SMEs on average failed to verbalize 57.1% of clinical knowledge steps, did not verbalize or demonstrate 68.3% and 9.5% of action steps, and did not verbalize or demonstrate 73.2% and 40.4% of decision steps, respectively. This study demonstrates that SMEs teaching indirect ophthalmoscopy by free recall may unintentionally omit important steps, suggesting that compared with free recall, CTA may generate more comprehensive and thus potentially more effective instructional materials for teaching technical skills in veterinary medicine.

What Matters Most to Residency and Intern Selection Committees in Veterinary Medicine?

A recent survey of members of residency selection committees for the American College of Veterinary Internal Medicine and American College of Veterinary Surgeons boards found letters of recommendation to be the most important factor when reviewing a resident's application followed by class rank as the second most important factor. These statistics indicate an interesting, but possibly troubling trend. This Letter to The Editor discusses the major problems concerning these findings and what residency program committees might consider as an alternative.

Comparing Entrustment and Competence: An Exploratory Look at Performance-Relevant Information in the Final Year of a Veterinary Program.

Workplace-based assessments and entrustment scales have two primary goals: providing formative information to assist students with future learning; and, determining if and when learners are ready for safe, independent practice. To date, there has not been an evaluation of the relationship between these performance-relevant information pieces in veterinary medicine. This study collected quantitative and qualitative data from a single cohort of final-year students (n = 27) across in-training evaluation reports (ITERs) and entrustment scales in a distributed veterinary hospital environment. Here we compare progression in scoring and performance within and across student, within and across method of assessment, over time. Narrative comments were quantified using the Completed Clinical Evaluation Report Rating (CCERR) instrument to assess quality of written comments. Preliminary evidence suggests that we may be capturing different aspects of performance using these two different methods. Specifically, entrustment scale scores significantly increased over time, while ITER scores did not. Typically, comments on entrustment scale scores were more learner specific, longer, and used more of a coaching voice. Longitudinal evaluation of learner performance is important for learning and demonstration of competence; however, the method of data collection could influence how feedback is structured and how performance is ultimately judged.

Using Muse: Rapid Mobile Assessment of Brain Performance.

The advent of mobile electroencephalography (mEEG) has created a means for large scale collection of neural data thus affording a deeper insight into cognitive phenomena such as cognitive fatigue. Cognitive fatigue - a neural state that is associated with an increased incidence of errorful performance - is responsible for accidents on a daily basis which at times can cost human lives. To gain better insight into the neural signature of cognitive fatigue in the present study we used mEEG to examine the relationship between perceived cognitive fatigue and human-event related brain potentials (ERPs) and electroencephalographic (EEG) oscillations in a sample of 1,000 people. As a secondary goal, we wanted to further demonstrate the capability of mEEG to accurately measure ERP and EEG data. To accomplish these goals, participants performed a standard visual oddball task on an Apple iPad while EEG data were recorded from a Muse EEG headband. Counter to traditional EEG studies, experimental setup and data collection was completed in less than seven minutes on average. An analysis of our EEG data revealed robust N200 and P300 ERP components and neural oscillations in the delta, theta, alpha, and beta bands. In line with previous findings we observed correlations between ERP components and EEG power and perceived cognitive fatigue. Further, we demonstrate here that a linear combination of ERP and EEG features is a significantly better predictor of perceived cognitive fatigue than any ERP or EEG feature on its own. In sum, our results provide validation of mEEG as a viable tool for research and provide further insight into the impact of cognitive fatigue on the human brain.

Development of and Validity Evidence for a Canine Ocular Model for Training Novice Veterinary Students to Perform a Fundic Examination.

Indirect fundoscopy is challenging for novice learners, as patients are often intolerant of the procedure, impeding development of proficiency. To address this, we developed a canine ocular simulator that we hypothesized would improve student learning compared to live dogs. Six board-certified veterinary ophthalmologists and 19 second-year veterinary students (novices) performed an indirect fundic examination on the model and live dog. Prior to assessment, novices were introduced to the skill with a standardized teaching protocol and practiced (without feedback) with either the model (n = 10) or live dog (n = 9) for 30 minutes. All participants evaluated realism and usefulness of the model using a Likert-type scale. Performance on the live dog and model was evaluated in all participants using time to completion of task, performance of fundic examination using a checklist and global score, identification of objects in the fundus of the model, and evaluation of time spent looking at the fundus of the model using eye tracking. Novices (trained on simulator or live dogs) were compared in fundic examination performance on the live dog and identification of shapes in the model. In general, experts performed the fundic examination faster (p ≤ .0003) and more proficiently than the novices, although there were no differences in eye tracking behavior between groups (p ≥ .06). No differences were detected between training on simulator versus live dog in development of fundoscopy skills in novices (p ≥ .20). These findings suggest that this canine model may be an effective tool to train students to perform fundoscopy.

The Impact of Covariates in Voxel-Wise Lesion-Symptom Mapping.

Background: Voxel-wise lesion-symptom mapping (VLSM) is a statistical technique to infer the structure-function relationship in patients with cerebral strokes. Previous VLSM research suggests that it is important to adjust for various confounders such as lesion size to minimize the inflation of true effects. The aim of this work is to investigate the regional impact of covariates on true effects in VLSM. Methods: A total of 222 follow-up datasets of acute ischemic stroke patients with known NIH Stroke Scale (NIHSS) score at 48-h post-stroke were available for this study. Patient age, lesion volume, and follow-up imaging time were tested for multicollinearity using variance inflation factor analysis and used as covariates in VLSM analyses. Covariate importance maps were computed from the VLSM results by standardizing the beta coefficients of general linear models. Results: Covariates were found to have distinct regional importance with respect to lesion eloquence in the brain. Age has a relatively higher importance in the superior temporal gyrus, inferior parietal lobule, and in the pre- and post-central gyri. Volume explains more variability in the opercular area of the insula, inferior frontal gyrus, and caudate. Follow-up imaging time accounts for most of the variance in the globus pallidus, ventromedial- and dorsolateral putamen, dorsal caudate, pre-motor thalamus, and the dorsal insula. Conclusions: This is the first study investigating and revealing distinctive regional patterns of importance for covariates typically used in VLSM. These covariate importance maps can improve our understanding of the lesion-deficit relationships in patients and could prove valuable for patient-specific treatment and rehabilitation planning.

Summative Assessment of Interprofessional "Collaborative Practice" Skills in Graduating Medical Students: A Validity Argument.

PURPOSE: To describe how the authors developed an objective structured clinical examination (OSCE) station to assess aspects of collaborative practice competency and how they then assessed validity using Kane's framework. METHOD: After piloting the collaborative practice OSCE station in 2015 and 2016, this was introduced at the Cumming School of Medicine in 2017. One hundred fifty-five students from the class of 2017 and 22 students from the class of 2018 participated. To create a validity argument, the authors used Kane's framework that views the argument for validity as 4 sequential inferences on the validity of scoring, generalization, extrapolation, and implications, RESULTS: Scoring validity is supported by psychometric analysis of checklist items and the fact that the contribution of rater specificity to students' ratings was similar to OSCE stations assessing clinical skills alone. The claim of validity of generalization is backed by structural equation modeling and confirmatory factor analysis that identified 5 latent variables, including 3 related to collaborative practice ("provides an effective handover," "provides mutual support," and "shares their mental model"). Validity of extrapolation is argued based upon the correlation between the rating for "shares their mental model" and the rating on in-training evaluations for "relationship with other members of the health care team," in addition to the association between performance on the collaborative practice OSCE station and the subsequent rating of performance during residency. Finally, validity of implications is supported by the fact that pass/fail decisions on the collaborative practice station were similar to other stations and by the observation that ratings on different aspects of collaborative practice associate with pass/fail decisions. CONCLUSIONS: Based upon the validity argument presented, the authors posit that this tool can be used to assess the collaborative practice competence of graduating medical students and the adequacy of training in collaborative practice.

Electroencephalography correlates of transcranial direct-current stimulation enhanced surgical skill learning: A replication and extension study.

Transcranial direct-current stimulation (tDCS), an increasingly applied form of non-invasive brain stimulation, can augment the acquisition of motor skills. Motor learning investigations of tDCS are limited to simple skills, where mechanisms are increasingly understood. Investigations of meaningful, complex motor skills possessed by humans, such as surgical skills, are limited. This replication and extension of our previous findings used electroencephalography (EEG) to determine how tDCS and complex surgical training alters electrical activity in the sensorimotor network to enhance complex surgical skill acquisition. In twenty-two participants, EEG was recorded during baseline performance of simulation-based laparoscopic surgical skills. Participants were randomized to receive 20 min of primary motor cortex targeting anodal tDCS or sham concurrent to 1 h of surgical skill training. EEG was reassessed following training, during a post-training repetition of the surgical tasks. Our results replicated our previous study suggesting that compared to sham, anodal tDCS enhanced the acquisition of unimanual surgical skill. Surgical training modulated delta frequency band activity in sensorimotor regions. Next, the performance of unimanual and bimanual skills evoked unique EEG profiles, primarily within the beta frequency-band in parietal regions. Finally, tDCS-paired surgical training independently modulated delta and alpha frequency-bands in sensorimotor regions. Application of tDCS during surgical skill training is feasible, safe and tolerable. In conclusion, we are the first to explore electrical brain activity during performance of surgical skills, how electrical activity may change during surgical training and how tDCS alters the brain to enhance skill acquisition. The results provide preliminary evidence of neural markers that can be targeted by neuromodulation to optimize complex surgical training.

Quantifying two-dimensional and three-dimensional stereoscopic learning in anatomy using electroencephalography.

Advances in computer visualization enabling both 2D and 3D representation have generated tools to aid perception of spatial relationships and provide a new forum for instructional design. A key knowledge gap is the lack of understanding of how the brain neurobiologically processes and learns from spatially presented content, and new quantitative variables are required to address this gap. The objective of this study was to apply quantitative neural measures derived from electroencephalography (EEG) to examine stereopsis in anatomy learning by comparing mean amplitude changes in N250 (related to object recognition) and reward positivity (related to responding to feedback) event related to potential components using a reinforcement-based learning paradigm. Health sciences students (n = 61) learned to identify and localize neuroanatomical structures using 2D, 3D, or a combination of models while EEG and behavioral (accuracy) data were recorded. Participants learning using 3D models had a greater object recognition (N250 amplitude) compared to those who learned from 2D models. Based on neurological results, interleaved learning incorporating both 2D and 3D models provided an advantage in learning, retention, and transfer activities represented by decreased reward positivity amplitude. Behavioral data did not have the same sensitivity as neural data for distinguishing differences in learning with and without stereopsis in these learning activities. Measuring neural activity reveals new insights in applied settings for educators to consider when incorporating stereoscopic models in the design of learning interventions.

Validity evidence for programmatic assessment in competency-based education.

INTRODUCTION: Competency-based education (CBE) is now pervasive in health professions education. A foundational principle of CBE is to assess and identify the progression of competency development in students over time. It has been argued that a programmatic approach to assessment in CBE maximizes student learning. The aim of this study is to investigate if programmatic assessment, i. e., a system of assessment, can be used within a CBE framework to track progression of student learning within and across competencies over time. METHODS: Three workplace-based assessment methods were used to measure the same seven competency domains. We performed a retrospective quantitative analysis of 327,974 assessment data points from 16,575 completed assessment forms from 962 students over 124 weeks using both descriptive (visualization) and modelling (inferential) analyses. This included multilevel random coefficient modelling and generalizability theory. RESULTS: Random coefficient modelling indicated that variance due to differences in inter-student performance was highest (40%). The reliability coefficients of scores from assessment methods ranged from 0.86 to 0.90. Method and competency variance components were in the small-to-moderate range. DISCUSSION: The current validation evidence provides cause for optimism regarding the explicit development and implementation of a program of assessment within CBE. The majority of the variance in scores appears to be student-related and reliable, supporting the psychometric properties as well as both formative and summative score applications.

Chasing the zone: Reduced beta power predicts baseball batting performance.

Mental state prior to sports skill execution is related to subsequent performance. For example, relationships between pre-performance electroencephalogram (EEG) power and subsequent movement outcomes in golf putting, pistol shooting, and basketball free throw shooting have been previously reported. With that said, the existing body of research examining the pre-performance EEG - performance relationship has been focused on the execution of internally as opposed to externally-paced motor skills. Given that the execution of internally and externally-paced movements are dependent on different neural pathways, in the present study we examined whether or not pre-performance EEG power predicted ensuing performance of an externally-paced motor skill - baseball batting. Sixty-seven baseball players had EEG data recorded for 120 s prior to batting practice. Performance was assessed by three expert coaches and the accuracy of coach performance ratings was verified via Generalizability Theory. An analysis of our data revealed an inverse relationship between frontal EEG power in the beta range and subsequent batting performance - reduced beta power was associated with better batting performance whereas increased beta power was associated with worse batting performance. Our results are in line with prior research that has demonstrated a relationship between increased EEG power in the beta range and the subsequent commitment of motor errors in addition to the aforementioned work examining pre-performance EEG and the execution of internally-paced motor skills.

Improving CPR quality with distributed practice and real-time feedback in pediatric healthcare providers - A randomized controlled trial.

OBJECTIVES: Guideline compliant CPR is associated with improved survival for patients with cardiac arrest. Conventional Basic Life Support (BLS) training results in suboptimal CPR competency and skill retention. We aimed to compare the effectiveness of distributed CPR training with real-time feedback to conventional BLS training for CPR skills in pediatric healthcare providers. METHODS: Healthcare providers were randomized into receiving annual BLS training (control) or distributed training with real-time feedback (intervention). The intervention group was asked to practice CPR for 2 min on mannequins while receiving real-time CPR feedback, at least once per month. Control group participants were not asked to practice CPR during the study period. Excellent CPR was defined as 90% guideline-compliance for depth, rate and recoil of chest compressions. CPR performance of participants was assessed (on infant and adult-sized mannequins) every 3 months for a duration of 12 months. CPR performance was compared between the 2 groups. RESULTS: A total of 87 healthcare providers were included in the analyses (control n = 41, intervention n = 46). Baseline assessment showed no significant difference in CPR performance across the 2 groups. The intervention group has a significantly greater proportion of participants with excellent CPR compared with the control group on an adult sized mannequin (14.6% vs. 54.3%, p < 0.001) and infant-sized mannequin (19.5% vs. 71.7%, p < 0.001) at the end of the study. In the intervention group, all CPR metrics except infant depth were improved and retained over the course of the study. CONCLUSION: Distributed CPR training with real-time feedback improves the compliance of AHA guidelines of quality of CPR.

Selection for family medicine residency training in Canada: How consistently are the same students ranked by different programs?

OBJECTIVE: To examine the consistency of the ranking of Canadian and US medical graduates who applied to Canadian family medicine (FM) residency programs between 2007 and 2013. DESIGN: Descriptive cross-sectional study. SETTING: Family medicine residency programs in Canada. PARTICIPANTS: All 17 Canadian medical schools allowed access to their anonymized program rank-order lists of students applying to FM residency programs submitted to the first iteration of the Canadian Resident Matching Service match from 2007 to 2013. MAIN OUTCOME MEASURES: The rank position of medical students who applied to more than 1 FM residency program on the rank-order lists submitted by the programs. Anonymized ranking data submitted to the Canadian Resident Matching Service from 2007 to 2013 by all 17 FM residency programs were used. Ranking data of eligible Canadian and US medical graduates were analyzed to assess the within-student and between-student variability in rank score. These covariance parameters were then used to calculate the intraclass correlation coefficient (ICC) for all programs. Program descriptions and selection criteria were also reviewed to identify sites with similar profiles for subset ICC analysis. RESULTS: Between 2007 and 2013, the consistency of ranking by all programs was fair at best (ICC = 0.34 to 0.39). The consistency of ranking by larger urban-based sites was weak to fair (ICC = 0.23 to 0.36), and the consistency of ranking by sites focusing on training for rural practice was weak to moderate (ICC = 0.16 to 0.55). CONCLUSION: In most cases, there is a low level of consistency of ranking of students applying for FM training in Canada. This raises concerns regarding fairness, particularly in relation to expectations around equity and distributive justice in selection processes.

A Reinforcement-Based Learning Paradigm Increases Anatomical Learning and Retention-A Neuroeducation Study.

In anatomy education, a key hurdle to engaging in higher-level discussion in the classroom is recognizing and understanding the extensive terminology used to identify and describe anatomical structures. Given the time-limited classroom environment, seeking methods to impart this foundational knowledge to students in an efficient manner is essential. Just-in-Time Teaching (JiTT) methods incorporate pre-class exercises (typically online) meant to establish foundational knowledge in novice learners so subsequent instructor-led sessions can focus on deeper, more complex concepts. Determining how best do we design and assess pre-class exercises requires a detailed examination of learning and retention in an applied educational context. Here we used electroencephalography (EEG) as a quantitative dependent variable to track learning and examine the efficacy of JiTT activities to teach anatomy. Specifically, we examined changes in the amplitude of the N250 and reward positivity event-related brain potential (ERP) components alongside behavioral performance as novice students participated in a series of computerized reinforcement-based learning modules to teach neuroanatomical structures. We found that as students learned to identify anatomical structures, the amplitude of the N250 increased and reward positivity amplitude decreased in response to positive feedback. Both on a retention and transfer exercise when learners successfully remembered and translated their knowledge to novel images, the amplitude of the reward positivity remained decreased compared to early learning. Our findings suggest ERPs can be used as a tool to track learning, retention, and transfer of knowledge and that employing the reinforcement learning paradigm is an effective educational approach for developing anatomical expertise.