Opportunities and Applications of Educational Technologies in Surgical Education and Assessment.

BACKGROUND: Surgical education is challenged by continuously increasing clinical content, greater subspecialization, and public scrutiny of access to high quality surgical care. Since the last Blue Ribbon Committee on surgical education, novel technologies have been developed including artificial intelligence and telecommunication. OBJECTIVES AND METHODS: The goals of this Blue Ribbon Sub-Committee were to describe the latest technological advances and construct a framework for applying these technologies to improve the effectiveness and efficiency of surgical education and assessment. An additional goal was to identify implementation frameworks and strategies for centers with different resources and access. All sub-committee recommendations were included in a Delphi consensus process with the entire Blue Ribbon Committee (N=67). RESULTS: Our sub-committee found several new technologies and opportunities that are well poised to improve the effectiveness and efficiency of surgical education and assessment (see Tables 1-3). Our top recommendation was that a Multidisciplinary Surgical Educational Council be established to serve as an oversight body to develop consensus, facilitate implementation, and establish best practices for technology implementation and assessment. This recommendation achieved 93% consensus during the first round of the Delphi process. CONCLUSION: Advances in technology-based assessment, data analytics, and behavioral analysis now allow us to create personalized educational programs based on individual preferences and learning styles. If implemented properly, education technology has the promise of improving the quality and efficiency of surgical education and decreasing the demands on clinical faculty.

Key Issues in Surgical Residency Education: Recommendations of the Blue Ribbon II Committee Residency Education Subcommittee.

In September 2022, a summit was convened by the American Board of Surgery (ABS) to discuss competency-based reform in surgical education. A key output of that summit was the recommendation that the prior work of the Blue Ribbon I Committee convened 20 years earlier be revived. With leadership from the American College of Surgeons (ACS) and the American Surgical Association (ASA) , the Blue Ribbon Committee (BRC) II was subsequently convened. This paper describes the output of the Residency Education Subcommittee of the BRC II Committee. The Subcommittee organized its work around prioritized themes including curriculum, assessment, and transition to practice. Top recommendations, time-based action steps, potential barriers, and required resources were detailed and vetted through group discussion, broader Committee review and critique, and subsequent refinement. Primary concluding emphases included transitioning to a competency-based training model, facilitating dynamically capable curricular reform emphasizing the digital transformation of surgical care, using predictive analytic assessment strategies to optimize training effectiveness and efficiency, and creating mentorship strategies to govern the transition from training to independent practice in an outcomes-accountable fashion. It was recognized that coordinated efforts across existing organizational structures will be required, informed by dataset integration strategies that meaningfully measure educational and related patient outcomes.

An American Board of Surgery Pilot of Video Assessment of Surgeon Technical Performance in Surgery.

OBJECTIVE: The American Board of Surgery (ABS) sought to investigate the suitability of video-based assessment (VBA) as an adjunct to certification for assessing technical skills. BACKGROUND: Board certification is based on the successful completion of a residency program coupled with knowledge and reasoning assessments. VBA is a new modality for evaluating operative skills that have been shown to correlate with patient outcomes after surgery. METHODS: Diplomates of the ABS were initially assessed for background knowledge and interest in VBA. Surgeons were then solicited to participate in the pilot. Three commercially available VBA platforms were identified and used for the pilot assessment. All participants served as reviewers and reviewees for videos. After the interaction, participants were surveyed regarding their experiences and recommendations to the ABS. RESULTS: To the initial survey, 4853/25,715 diplomates responded. The majority were neither familiar with VBA, nor the tools used for operative assessments. Two hundred seventy-four surgeons actively engaged in the subsequent pilot. One hundred sixty-nine surgeons completed the postpilot survey. Most participants found the process straightforward. Of the participants, 74% felt that the feedback would help their surgical practice. The majority (81%) remain interested in VBA for continuing medical education credits. Using VBA in continuous certification could improve surgeon skills felt by 70%. Two-thirds of participants felt VBA could help identify and remediate underperforming surgeons. Identified barriers to VBA included limitations for open surgery, privacy issues, and technical concerns. CONCLUSIONS: VBA is promising as an adjunct to the current board certification process and should be further considered by the ABS.

Surgical Education.

This paper summarizes the proceedings of the joint European Surgical Association ESA/American Surgical Association symposium on Surgical Education that took place in Bordeaux, France, as part of the celebrations for 30 years of ESA scientific meetings. Three presentations on the use of quantitative metrics to understand technical decisions, coaching during training and beyond, and entrustable professional activities were presented by American Surgical Association members and discussed by ESA members in a symposium attended by members of both associations.

Generating Rare Surgical Events Using CycleGAN: Addressing Lack of Data for Artificial Intelligence Event Recognition.

INTRODUCTION: Artificial Intelligence (AI) has shown promise in facilitating surgical video review through automatic recognition of surgical activities/events. There are few public video data sources that demonstrate critical yet rare events which are insufficient to train AI for reliable video event recognition. We suggest that a generative AI algorithm can create artificial massive bleeding images for minimally invasive lobectomy that can be used to augment the current lack of data in this field. MATERIALS AND METHODS: A generative adversarial network (GAN) algorithm was used (CycleGAN) to generate artificial massive bleeding event images. To train CycleGAN, six videos of minimally invasive lobectomies were utilized from which 1819 frames of nonbleeding instances and 3178 frames of massive bleeding instances were used. RESULTS: The performance of the CycleGAN algorithm was tested on a new video that was not used during the training process. The trained CycleGAN was able to alter the laparoscopic lobectomy images according to their corresponding massive bleeding images, where the contents of the original images were preserved (e.g., location of tools in the scene) and the style of each image is changed to massive bleeding (i.e., blood automatically added to appropriate locations on the images). CONCLUSIONS: The result could suggest a promising approach to supplement the lack of data for the rare massive bleeding event that can occur during minimally invasive lobectomy. Future work could be dedicated to developing AI algorithms to identify surgical strategies and actions that potentially lead to massive bleeding and warn surgeons prior to this event occurrence.

AI-Based Video Segmentation: Procedural Steps or Basic Maneuvers?

INTRODUCTION: Video-based review of surgical procedures has proven to be useful in training by enabling efficiency in the qualitative assessment of surgical skill and intraoperative decision-making. Current video segmentation protocols focus largely on procedural steps. Although some operations are more complex than others, many of the steps in any given procedure involve an intricate choreography of basic maneuvers such as suturing, knot tying, and cutting. The use of these maneuvers at certain procedural steps can convey information that aids in the assessment of the complexity of the procedure, surgical preference, and skill. Our study aims to develop and evaluate an algorithm to identify these maneuvers. METHODS: A standard deep learning architecture was used to differentiate between suture throws, knot ties, and suture cutting on a data set comprised of videos from practicing clinicians (N = 52) who participated in a simulated enterotomy repair. Perception of the added value to traditional artificial intelligence segmentation was explored by qualitatively examining the utility of identifying maneuvers in a subset of steps for an open colon resection. RESULTS: An accuracy of 84% was reached in differentiating maneuvers. The precision in detecting the basic maneuvers was 87.9%, 60%, and 90.9% for suture throws, knot ties, and suture cutting, respectively. The qualitative concept mapping confirmed realistic scenarios that could benefit from basic maneuver identification. CONCLUSIONS: Basic maneuvers can indicate error management activity or safety measures and allow for the assessment of skill. Our deep learning algorithm identified basic maneuvers with reasonable accuracy. Such models can aid in artificial intelligence-assisted video review by providing additional information that can complement traditional video segmentation protocols.

SAGES consensus recommendations on surgical video data use, structure, and exploration (for research in artificial intelligence, clinical quality improvement, and surgical education).

BACKGROUND: Surgery generates a vast amount of data from each procedure. Particularly video data provides significant value for surgical research, clinical outcome assessment, quality control, and education. The data lifecycle is influenced by various factors, including data structure, acquisition, storage, and sharing; data use and exploration, and finally data governance, which encompasses all ethical and legal regulations associated with the data. There is a universal need among stakeholders in surgical data science to establish standardized frameworks that address all aspects of this lifecycle to ensure data quality and purpose. METHODS: Working groups were formed, among 48 representatives from academia and industry, including clinicians, computer scientists and industry representatives. These working groups focused on: Data Use, Data Structure, Data Exploration, and Data Governance. After working group and panel discussions, a modified Delphi process was conducted. RESULTS: The resulting Delphi consensus provides conceptualized and structured recommendations for each domain related to surgical video data. We identified the key stakeholders within the data lifecycle and formulated comprehensive, easily understandable, and widely applicable guidelines for data utilization. Standardization of data structure should encompass format and quality, data sources, documentation, metadata, and account for biases within the data. To foster scientific data exploration, datasets should reflect diversity and remain adaptable to future applications. Data governance must be transparent to all stakeholders, addressing legal and ethical considerations surrounding the data. CONCLUSION: This consensus presents essential recommendations around the generation of standardized and diverse surgical video databanks, accounting for multiple stakeholders involved in data generation and use throughout its lifecycle. Following the SAGES annotation framework, we lay the foundation for standardization of data use, structure, and exploration. A detailed exploration of requirements for adequate data governance will follow.

Do Individual Surgeon Preferences Affect Procedural Outcomes?

OBJECTIVES: Surgeon preferences such as instrument and suture selection and idiosyncratic approaches to individual procedure steps have been largely viewed as minor differences in the surgical workflow. We hypothesized that idiosyncratic approaches could be quantified and shown to have measurable effects on procedural outcomes. METHODS: At the American College of Surgeons (ACS) Clinical Congress, experienced surgeons volunteered to wear motion tracking sensors and be videotaped while evaluating a loop of porcine intestines to identify and repair 2 preconfigured, standardized enterotomies. Video annotation was used to identify individual surgeon preferences and motion data was used to quantify surgical actions. χ 2 analysis was used to determine whether surgical preferences were associated with procedure outcomes (bowel leak). RESULTS: Surgeons' (N=255) preferences were categorized into 4 technical decisions. Three out of the 4 technical decisions (repaired injuries together, double-layer closure, corner-stitches vs no corner-stitches) played a significant role in outcomes, P <0.05. Running versus interrupted did not affect outcomes. Motion analysis revealed significant differences in average operative times (leak: 6.67 min vs no leak: 8.88 min, P =0.0004) and work effort (leak-path length=36.86 cm vs no leak-path length=49.99 cm, P =0.001). Surgeons who took the riskiest path but did not leak had better bimanual dexterity (leak=0.21/1.0 vs no leak=0.33/1.0, P =0.047) and placed more sutures during the repair (leak=4.69 sutures vs no leak=6.09 sutures, P =0.03). CONCLUSIONS: Our results show that individual preferences affect technical decisions and play a significant role in procedural outcomes. Future analysis in more complex procedures may make major contributions to our understanding of contributors to procedure outcomes.

From Listening to Action: Academic Surgical Departmental Response to Social Injustice Through Curricular Development.

OBJECTIVE: The aim of this study was to describe the development and evaluation of a structured department wide cultural competency curriculum. SUMMARY BACKGROUND DATA: Despite numerous organizational policies and statements, social injustice and bias still exist. Our department committed to assist individuals of the entire department to develop foundational knowledge and skills to combat implicit bias and systemic racism through the creation of a cultural competency curriculum. The purpose of this manuscript is to detail our curriculum and the evaluation of its effectiveness. METHODS: Using a well-established curriculum development framework, a cultural competency curriculum was developed focusing on knowledge, skills and attitudes at the individual level, for all members of the department. The curriculum was implemented through 6-hour-long sessions over a 9-week period. Effectiveness was assessed through a post curriculum survey. RESULTS: Twenty percent of the respondents had experienced bias based on race, ethnicity, or sexual orientation in the past 12 months, whereas 30% had experienced bias based on sex. Seventy-one percent independently explored related topics. The curriculum was overall well received and generally achieved the goals and objectives. CONCLUSION: Using a standard curriculum development framework, an effective department-wide cultural competency curriculum can be developed and implemented.

The Role of Race and Gender in the Career Experiences of Black/African American Academic Surgeons: A Survey of the Society of Black Academic Surgeons and a Call to Action.

OBJECTIVE: To determine the role of race and gender in the career experience of Black/AA academic surgeons and to quantify the prevalence of experience with racial and gender bias stratified by gender. SUMMARY OF BACKGROUND DATA: Compared to their male counterparts, Black/African American women remain significantly underrepresented among senior surgical faculty and department leadership. The impact of racial and gender bias on the academic and professional trajectory of Black/AA women surgeons has not been well-studied. METHODS: A cross-sectional survey regarding demographics, employment, and perceived barriers to career advancement was distributed via email to faculty surgeon members of the Society of Black American Surgeons (SBAS) in September 2019. RESULTS: Of 181 faculty members, 53 responded (29%), including 31 women (58%) and 22 men (42%). Academic positions as a first job were common (men 95% vs women 77%, P = 0.06). Men were more likely to attain the rank of full professor (men 41% vs women 7%, P = 0.01). Reports of racial bias in the workplace were similar (women 84% vs men 86%, not significant); however, reports of gender bias (women 97% vs men 27%, P < 0.001) and perception of salary inequities (women 89% vs 63%, P = 0.02) were more common among women. CONCLUSIONS AND RELEVANCE: Despite efforts to increase diversity, high rates of racial bias persist in the workplace. Black/AA women also report experiencing a high rate of gender bias and challenges in academic promotion.

Can Deep Learning Algorithms Help Identify Surgical Workflow and Techniques?

BACKGROUND: Surgical videos are now being used for performance review and educational purposes; however, broad use is still limited due to time constraints. To make video review more efficient, we implemented Artificial Intelligence (AI) algorithms to detect surgical workflow and technical approaches. METHODS: Participants (N = 200) performed a simulated open bowel repair. The operation included two major phases: (1) Injury Identification and (2) Suture Repair. Accordingly, a phase detection algorithm (MobileNetV2+GRU) was implemented to automatically detect the two phases using video data. In addition, participants were noted to use three different technical approaches when running the bowel: (1) use of both hands, (2) use of one hand and one tool, or (3) use of two tools. To discern the three technical approaches, an object detection (YOLOv3) algorithm was implemented to recognize objects that were commonly used during the Injury Identification phase (hands versus tools). RESULTS: The phase detection algorithm achieved high precision (recall) when segmenting the two phases: Injury Identification (86 ± 9% [81 ± 12%]) and Suture Repair (81 ± 6% [81 ± 16%]). When evaluating three technical approaches in running the bowel, the object detection algorithm achieved high average precisions (Hands [99.32%] and Tools [94.47%]). The three technical approaches showed no difference in execution time (Kruskal-Wallis Test: P= 0.062) or injury identification (not missing an injury) (Chi-squared: P= 0.998). CONCLUSIONS: The AI algorithms showed high precision when segmenting surgical workflow and identifying technical approaches. Automation of these techniques for surgical video databases has great potential to facilitate efficient performance review.

Performance assessment using sensor technology.

Over the past 30 years, there have been numerous, noteworthy successes in the development, validation, and implementation of clinical skills assessments. Despite this progress, the medical profession has barely scratched the surface towards developing assessments that capture the true complexity of hands-on skills in procedural medicine. This paper highlights the development implementation and new discoveries in performance metrics when using sensor technology to assess cognitive and technical aspects of hands-on skills.

SAGES consensus recommendations on an annotation framework for surgical video.

BACKGROUND: The growing interest in analysis of surgical video through machine learning has led to increased research efforts; however, common methods of annotating video data are lacking. There is a need to establish recommendations on the annotation of surgical video data to enable assessment of algorithms and multi-institutional collaboration. METHODS: Four working groups were formed from a pool of participants that included clinicians, engineers, and data scientists. The working groups were focused on four themes: (1) temporal models, (2) actions and tasks, (3) tissue characteristics and general anatomy, and (4) software and data structure. A modified Delphi process was utilized to create a consensus survey based on suggested recommendations from each of the working groups. RESULTS: After three Delphi rounds, consensus was reached on recommendations for annotation within each of these domains. A hierarchy for annotation of temporal events in surgery was established. CONCLUSIONS: While additional work remains to achieve accepted standards for video annotation in surgery, the consensus recommendations on a general framework for annotation presented here lay the foundation for standardization. This type of framework is critical to enabling diverse datasets, performance benchmarks, and collaboration.

Situating Artificial Intelligence in Surgery: A Focus on Disease Severity.

OBJECTIVES: Artificial intelligence (AI) has numerous applications in surgical quality assurance. We assessed AI accuracy in evaluating the critical view of safety (CVS) and intraoperative events during laparoscopic cholecystectomy. We hypothesized that AI accuracy and intraoperative events are associated with disease severity. METHODS: One thousand fifty-one laparoscopic cholecystectomy videos were annotated by AI for disease severity (Parkland Scale), CVS achievement (Strasberg Criteria), and intraoperative events. Surgeons performed focused video review on procedures with ≥1 intraoperative events (n = 335). AI versus surgeon annotation of CVS components and intraoperative events were compared. For all cases (n = 1051), intraoperative-event association with CVS achievement and severity was examined using ordinal logistic regression. RESULTS: Using AI annotation, surgeons reviewed 50 videos/hr. CVS was achieved in ≤10% of cases. Hepatocystic triangle and cystic plate visualization was achieved more often in low-severity cases (P < 0.03). AI-surgeon agreement for all CVS components exceeded 75%, with higher agreement in high-severity cases (P < 0.03). Surgeons agreed with 99% of AI-annotated intraoperative events. AI-annotated intraoperative events were associated with both disease severity and number of CVS components not achieved. Intraoperative events occurred more frequently in high-severity versus low-severity cases (0.98 vs 0.40 events/case, P < 0.001). CONCLUSIONS: AI annotation allows for efficient video review and is a promising quality assurance tool. Disease severity may limit its use and surgeon oversight is still required, especially in complex cases. Continued refinement may improve AI applicability and allow for automated assessment.

Benchmarking Accomplishments of Leaders in American Surgery and Justification for Enhancing Diversity and Inclusion.

OBJECTIVE: To comprehensively assess the level of achievement and demographics of national surgical society presidents. BACKGROUND: Data on the accomplishments needed to rise to positions of national surgical leadership is scarce and merit alone does not always yield such opportunities. Recognizing the shortcomings of sex and ethnic diversity within academic surgical leadership, the American College of Surgeon (ACS), American Surgical Association (ASA), Association of Women Surgeons (AWS), and the Society of Black Academic Surgeons (SBAS) partnered to address these challenges by performing a comprehensive assessment of their presidents over the last 16 years. METHODS: ACS, ASA, AWS, and SBAS presidents' CVs, at the time of their presidential term, were assessed for demographics and scholastic achievements. Regression analyses controlling for age were performed to determine relative differences across societies. RESULTS: A total of 62 of the 64 presidents' CVs were received and assessed (97% response rate). There was a large discrepancy in the average age in years of ACS (70) and ASA (66) presidents compared to the AWS (51) and SBAS (53) presidents. For the ACS and ASA cohort, 87% were male and 83% were White, collectively. After controlling for age (52), the AWS and SBAS presidents' scholastic achievements were comparable to the ACS (and ASA) cohort in 9 and 12 of the 15 accessed metrics, respectively. CONCLUSION: The ACS and ASA presidents' CVs displayed unsurpassed scholastic achievement, and although not equivalent, both the AWS and the SBAS presidents had comparable attainment. These findings further substantiate that women and ethnic minority surgeons are deserving of additional national leadership consideration as organized medicine pursues a more diverse and reflective physician workforce.

A Call to Action: Black/African American Women Surgeon Scientists, Where are They?

OBJECTIVE: To determine the representation of Black/AA women surgeons in academic medicine among U.S. medical school faculty and to assess the number of NIH grants awarded to Black/AA women surgeon-scientists over the past 2 decades. SUMMARY OF BACKGROUND DATA: Despite increasing ethnic/racial and sex diversity in U.S. medical schools and residencies, Black/AA women have historically been underrepresented in academic surgery. METHODS: A retrospective review of the Association of American Medical Colleges 2017 Faculty Roster was performed and the number of grants awarded to surgeons from the NIH (1998-2017) was obtained. Data from the Association of American Medical Colleges included the total number of medical school surgery faculty, academic rank, tenure status, and department Chair roles. Descriptive statistics were performed. RESULTS: Of the 15,671 U.S. medical school surgical faculty, 123 (0.79%) were Black/AA women surgeons with only 11 (0.54%) being tenured faculty. When stratified by academic rank, 15 (12%) Black/AA women surgeons were instructors, 73 (59%) were assistant professors, 19 (15%) were associate professors, and 10 (8%) were full professors of surgery. Of the 372 U.S. department Chairs of surgery, none were Black/AA women. Of the 9139 NIH grants awarded to academic surgeons from 1998 and 2017, 31 (0.34%) grants were awarded to fewer than 12 Black/AA women surgeons. CONCLUSION: A significant disparity in the number of Black/AA women in academic surgery exists with few attaining promotion to the rank of professor with tenure and none ascending to the role of department Chair of surgery. Identifying and removing structural barriers to promotion, NIH grant funding, and academic advancement of Black/AA women as leaders and surgeon-scientists is needed.

Can Virtual Reality Be Used to Track Skills Decay During the Research Years?

BACKGROUND: Time away from surgical practice can lead to skills decay. Research residents are thought to be prone to skills decay, given their limited experience and reduced exposure to clinical activities during their research training years. This study takes a cross-sectional approach to assess differences in residents' skills at the beginning and end of their research years using virtual reality. We hypothesized that research residents will have measurable decay in psychomotor skills when evaluated using virtual reality. METHODS: Surgical residents (n = 28) were divided into two groups; the first group was just beginning their research time (clinical residents: n = 19) and the second group (research residents: n = 9) had just finished at least 2 y of research. All participants were asked to perform a target-tracking task using a haptic device, and their performance was compared using Welch's t-test. RESULTS: Research residents showed a higher level of "tracking error" (1.69 ± 0.44 cm versus 1.40 ± 0.19 cm; P = 0.04) and a similar level of "path length" (62.5 ± 10.5 cm versus 62.1 ± 5.2 cm; P = 0.92) when compared with clinical residents. CONCLUSIONS: The increased "tracking error" among residents at the end of their research time suggests fine psychomotor skills decay in residents who spend time away from clinical duties during laboratory time. This decay demonstrates the need for research residents to regularly participate in clinical activities, simulation, or assessments to minimize and monitor skills decay while away from clinical practice. Additional longitudinal studies may help better map learning and decay curves for residents who spend time away from clinical practice.

Modeling Surgical Technical Skill Using Expert Assessment for Automated Computer Rating.

OBJECTIVE: Computer vision was used to predict expert performance ratings from surgeon hand motions for tying and suturing tasks. SUMMARY BACKGROUND DATA: Existing methods, including the objective structured assessment of technical skills (OSATS), have proven reliable, but do not readily discriminate at the task level. Computer vision may be used for evaluating distinct task performance throughout an operation. METHODS: Open surgeries was videoed and surgeon hands were tracked without using sensors or markers. An expert panel of 3 attending surgeons rated tying and suturing video clips on continuous scales from 0 to 10 along 3 task measures adapted from the broader OSATS: motion economy, fluidity of motion, and tissue handling. Empirical models were developed to predict the expert consensus ratings based on the hand kinematic data records. RESULTS: The predicted versus panel ratings for suturing had slopes from 0.73 to 1, and intercepts from 0.36 to 1.54 (Average R2 = 0.81). Predicted versus panel ratings for tying had slopes from 0.39 to 0.88, and intercepts from 0.79 to 4.36 (Average R2 = 0.57). The mean square error among predicted and expert ratings was consistently less than the mean squared difference among individual expert ratings and the eventual consensus ratings. CONCLUSIONS: The computer algorithm consistently predicted the panel ratings of individual tasks, and were more objective and reliable than individual assessment by surgical experts.

Dynamic Visual Feedback During Junctional Tourniquet Training.

BACKGROUND: This project involved the development and evaluation of a new visual bleeding feedback (VBF) system for tourniquet training. We hypothesized that dynamic VBF during junctional tourniquet training would be helpful and well received by trainees. MATERIALS AND METHODS: We designed the VBF to simulate femoral bleeding. Medical students (n = 15) and emergency medical service (EMS) members (n = 4) were randomized in a single-blind, crossover study to the VBF or without feedback groups. Poststudy surveys assessing VBF usefulness and recommendations were conducted along with participants' reported confidence using a 7-point Likert scale. Data from the different groups were compared using Wilcoxon signed-rank and rank-sum tests. RESULTS: Participants rated the helpfulness of the VBF highly (6.53/7.00) and indicated they were very likely to recommend the VBF simulator to others (6.80/7.00). Pre- and post-VBF confidence were not statistically different (P = 0.59). Likewise, tourniquet application times for VBF and without feedback before crossover were not statistically different (P = 0.63). Although participant confidence did not change significantly from beginning to end of the study (P = 0.46), application time was significantly reduced (P = 0.001). CONCLUSIONS: New tourniquet learners liked our VBF prototype and found it useful. Although confidence did not change over the course of the study for any group, application times improved. Future studies using outcomes of this study will allow us to continue VBF development as well as incorporate other quantitative measures of task performance to elucidate VBF's true benefit and help trainees achieve mastery in junctional tourniquet skills.