A multi-institutional study from the US ROPE Consortium examining factors associated with directly entering practice upon residency graduation.

BACKGROUND: There is concern regarding the competency of today's general surgery graduates as a large proportion defer independent practice in favor of additional fellowship training. Little is known about the graduates who directly enter general surgery practice and if their operative experiences during residency differ from graduates who pursue fellowship. METHODS: Nineteen Accreditation Council for Graduate Medical Education-accredited general surgery programs from the US Resident OPerative Experience Consortium were included. Demographics, career choice, and case logs from graduates between 2010 to 2020 were analyzed. RESULTS: There were 1,264 general surgery residents who graduated over the 11-year period. A total of 248 (19.6%) went directly into practice and 1,016 (80.4%) pursued fellowship. Graduates directly entering practice were more likely to be a high-volume resident (43.1% vs 30.5%, P < .01) and graduate from a high-volume program (49.2% vs 33.0%, P < .01). Direct-to-practice graduates performed 53 more cases compared with fellowship-bound graduates (1,203 vs 1,150, P < .01). On multivariable analysis, entering directly into practice was positively associated with total surgeon chief case volume (odds ratio = 1.47, 95% confidence interval 1.18-1.84, P < .01) and graduating from a US medical school (odds ratio = 2.54, 95% confidence interval 1.45-4.44, P < .01) while negatively associated with completing a dedicated research experience (odds ratio = 0.31, 95% confidence interval 0.22-0.45, P < .01). CONCLUSION: This is the first multi-institutional study exploring resident operative experience and career choice. These data suggest residents who desire immediate practice can tailor their experience with less research time and increased operative volume. These data may be helpful for programs when designing their experience for residents with different career goals.

Anatomy Scholars Program for Medical Students Entering a Surgical Residency.

A near-peer teaching experience for upper-level medical students could help prepare them for surgical residency by providing specific education opportunities and exposure to the field of surgery. Five medical students were selected to be near-peer teachers (NPTs) in gross anatomy, and then they reflected on their experiences. The NPTs spent the majority of effort in a teaching role, and reported improved NTS, anatomy knowledge, and dissection skills. MS1s and faculty also reported on the value of the ASP. Further development and evaluation of the ASP may be an excellent opportunity for future surgeons.

A Transitional Year Residency Program Provides Innovative Solutions for Unmatched Medical Students.

BACKGROUND: The transition to residency is competitive with more medical students applying for residency positions than slots available, and some will face challenges securing a position in their desired specialty. Our institution created a transitional year (TY) residency program in 2016 to help meet the needs of our medical students who did not initially secure a position in the main residency Match. OBJECTIVE: This report provides a brief overview of the TY program and analysis of the program's value from the inaugural 3 years (2017-2020). METHODS: The TY program is based at a midsized, urban, academic health center and features a tailorable curriculum emphasizing preparation for residents' specialty career plans. We used participatory action research and appreciative inquiry strategies as part of the annual program evaluation to examine TY residents' perceptions of the program's value. Stakeholder perceptions were also elicited from a purposive selection of 4 program directors and 2 key medical school education leaders. RESULTS: Internal evaluations revealed a high rate of resident satisfaction with the TY program and self-reported benefits such as increased confidence, clinical proficiency, and professional enculturation. Stakeholders valued the program as a potential pipeline for increasing physicians in the state and providing valuable direction to students' career trajectories. CONCLUSIONS: Creating a TY residency program to meet the needs of unmatched medical students was feasible to implement, acceptable to residents in meeting their academic and career needs, and provided a sustainable institutional solution with benefits to multiple stakeholders.

Psychometric Properties of the Copenhagen Burnout Inventory in an Academic Healthcare Institution Sample in the U.S.

The Copenhagen Burnout Inventory (CBI) has demonstrated good psychometric properties among respondents in many different countries, but minimal research exists using the CBI in a U.S.-based sample. The current study represents a secondary analysis of existing CBI data from 1,679 academic health center employees at one mid-size teaching hospital in the southeastern region of the U.S. Analyses assessed CBI scale reliability, confirmatory factorial validity, discriminant validity against a measure of meaningful work, and test invariance for professional role sub-groups (physicians, nurses/physician assistants, and other hospital staff), gender groups, and different age groups. Results provided evidence for good reliability and discriminant validity as well as construct validity supporting the CBI proposed three-factor structure. Configural and metric variance equivalence were demonstrated across the range of employee types, and across age and gender groups. Scalar invariance equivalence was not established, suggesting further research may be needed to support group mean comparisons using the CBI.

What makes them different? An exploration of mentoring for female faculty, residents, and medical students pursuing a career in surgery.

BACKGROUND: This qualitative study examines the roles of mentoring and gender in choosing and continuing in a surgical career for women across the continuum. METHODS: Semi-structured interviews were held with a purposive sample of 24 female surgical faculty, residents, and aspiring medical students from one institution between November 2018 and January 2019. Interview transcripts were analyzed using traditional thematic analysis methods aided by computerized software. RESULTS: The use of a mosaic approach in seeking mentoring to match one's personal and career-relevant support needs was described frequently. Same-gender role models were more important for early career women, while leadership mentoring and coaching were more desired by later career women. Gender differences in mentoring were identified but some of these differences may apply equally to women and men. CONCLUSIONS: Study findings contribute mentoring insights relevant to both women and men interested in pursuing and thriving in surgical careers.

A One-Year Institutional View of Resident Physician Burnout.

OBJECTIVES: This prospective study explores the prevalence, associated characteristics, and trajectory of burnout over one academic year in a multidisciplinary sample of resident physicians using a relatively new burnout survey instrument. METHODS: All residents from a U.S. academic health center (n = 633) were invited to complete the Copenhagen Burnout Inventory (CBI) three times, with 4-month time lags between invitations. A total of 281 (44%) provided complete CBI survey responses at least once, and 43 (7%) did at all three times. Descriptive statistics, cross-sectional analyses, correlations, and multivariable linear regression analyses were computed, as well as repeated measures ANOVAs and paired t tests, as appropriate, for each CBI domain (personal, work, patient-related burnout). RESULTS: About half had CBI scores indicating moderate-to-high levels of personal burnout (49-52%) and work-related burnout (45-49%), whereas patient-related burnout was less common (14-24%). However, patient-related burnout increased significantly from the beginning to the end of the year. Regression analyses indicated patient-related burnout was significantly higher for postgraduate year 1-2 residents compared to PGY 4+ residents, but was not significantly different by gender. Personal and work burnout scores were significantly higher for females. Persistently high burnout was observed in only 6% of respondents. CONCLUSIONS: In this study of resident physicians using the CBI, burnout was prevalent and higher levels of burnout were observed for females on the personal and work burnout domains, while junior residents had higher patient-related burnout. Persistently, high burnout was rare. The CBI demonstrated high reliability, was practical to administer, and produced similar results with existing burnout research.

Differences in burnout prevalence between clinical professionals and biomedical scientists in an academic medical centre: a cross-sectional survey.

OBJECTIVE: To determine the prevalence and associated factors for personal, work-related and patient/client-related burnout in clinical professionals and biomedical scientists in academic medicine. DESIGN: Prevalence survey using the Copenhagen Burnout Inventory. SETTING: Mid-size academic health centre. PARTICIPANTS: Clinical providers (n=6489) and biomedical scientists (n=248) were invited to complete the survey. 1646 completed responses (response rate 24.4%) were analysed. PRIMARY AND SECONDARY OUTCOME MEASURES: Prevalence estimates and adjusted ORs (AOR) were stratified for gender, age and professional category. RESULTS: Type of burnout varies across professional categories, with significant differences between clinicians and scientists. The prevalence of personal burnout was 52.7% (95%CI 50% to 55%), work-related burnout 47.5% (95%CI 45% to 49%) and patient/client-related burnout 20.3% (95%CI 18% to 22%). The prevalence of personal and work-related burnout was higher among women, while those aged 20-30 had a higher prevalence of all three burnout categories. Overall, clinical professionals had higher personal and work-related burnout, while biomedical scientists had higher client-related burnout. Accounting for the effects of gender and age, a significantly higher risk for personal burnout was found for physicians (AOR 1.64; 95%CI 1.3 to 2.1) and nurses (AOR 1.5; 95%CI 1.03 to 2.2). Significantly higher odds of work-related burnout were found for nurses (AOR 1.5; 95%CI 1.2 to 1.9) and residents (AOR 1.9; 95%CI 1.04 to 3.6). Basic scientists (AOR 10.0; 95%CI 5.7 to 17.6), physicians (AOR 2.8; 95%CI 1.9 to 4.1) and nurses (AOR 2.1; 95%CI 1.3 to 3.5) had higher odds of patient/client-related burnout. CONCLUSIONS: Types of burnout are unevenly distributed in academic medical centres. Physicians have higher risk of personal and patient/client-related burnout, residents have higher risk of work-related burnout, basic scientists are at higher risk of client-related burnout and nurses have higher odds of all three types of burnout. Interventions addressing the problem of burnout in clinical environments may be inadequate to support biomedical scientists.

Simulation-based training in urology residency programmes in the USA: Results of a nationwide survey.

OBJECTIVE: To evaluate the current usage of simulation in urological education in the USA and the barriers to incorporating a simulation-based educational curriculum, as the shift towards competency-based medical education has necessitated the introduction of simulation for training and assessing both non-technical and technical skills. MATERIALS AND METHODS: Residency programme directors at Accreditation Council for Graduate Medical Education (ACGME)-accredited urology training programmes in the USA were invited to respond to an anonymous electronic survey. The study evaluated the programme directors' experiences and opinions for the current usage of existing urology simulators. The survey also elicited receptiveness and the barriers for incorporating simulation-based training curricula within urology training programmes. RESULTS: In all, 43 completed surveys were received (35% response rate). Amongst responders, 97% (42/43) reported having access to a simulation education centre, and 60% (25/42) have incorporated simulation into their curriculum. A total of 87% (37/43) agreed that there is a role for a standardised simulator training curriculum, and 75% (30/40) agreed that simulators would improve operating room performance. A total of 64% (27/42) agreed that cost was a limiting factor, 12% (5/42) agreed on the cost-effectiveness of simulators, 35% (17/41) agreed there was an increased need for simulator education within work-hour limitations, and 38% (16/42) agreed a simulation programme would reduce patient risks and complications. CONCLUSIONS: The majority of urology programme directors consider that there is a role for incorporating a simulation-based curriculum into urology training. Barriers to implementation include cost burden, need for constant technology updates, need for advanced planning, and willingness of faculty to participate in administration.

National Landscape of General Surgery Mock Oral Examination Practices: Survey of Residency Program Directors.

INTRODUCTION: Mock oral examinations (MOEs) are used within surgery residency programs to prepare trainees for the American Board of Surgery (ABS) Certifying Exam (CE), but little work exists to guide programs in terms of best practices for implementing a general surgery MOE program. This study, endorsed by the Association for Program Directors in Surgery (APDS) Research Committee, aimed to better understand the national scope of current practices for general surgery MOEs. METHODS: General surgery residency program directors (PDs) were invited via the APDS listserv to complete a 27-item survey about their perceptions of the importance and correlates of MOEs, how their exams are structured, implementation barriers, and recent revisions to their MOE program. RESULTS: Of 98 PDs responding to the survey, 94% (n = 92) responded about the characteristics of their formal MOE programs. The majority required upper level resident participation and held the exams 2 to 3 times annually; far fewer involved lower level residents. Most programs structure their MOEs to mimic the CE format with 3 exam rooms (76%), using premade questions (66%), presenting 4 scenarios per room (59%), and using two examiners per room (85%). Most PDs (88%) believed MOEs were very important or essential for surgery trainees, which correlated with their ratings of how important MOEs are to their Clinical Competency Committee for determining resident advancement (r = 0.32, p < 0.002). Common barriers for implementing MOEs were availability of examiners and scenarios. About half indicated making recent or ongoing revisions to improve their MOEs. Many PDs indicated interest in collaborating regionally or nationally on MOE initiatives. CONCLUSIONS: MOEs were largely regarded as a highly valuable tool by PDs to prepare trainees for the general surgery CE. The majority of programs in this study provide a testing experience as similar to the CE as possible, although some variability in the structure of MOEs was identified. PDs also reported significant implementation barriers and a desire for more MOE collaboration.

Are Surgical Milestone Assessments Predictive of In-Training Examination Scores?

OBJECTIVES: With the recent utilization of Accreditation Council for Graduate Medical Education developmental milestones as part of resident evaluation, we sought to explore whether milestone-based ratings were associated with American Board of Surgery In-Training Examination (ABSITE) scores. METHODS: Mid-year milestone ratings were obtained from the Accreditation Council for Graduate Medical Education Accreditation Database System for years 2014, 2015, and 2016 for all postgraduate years 1-5 general surgery residents in our program and paired with ABSITE scores (n = 69) from January of the following year. Linear regression was used to assess predictor importance of milestones on both ABSITE percentage correct scores and ABSITE percentile scores. RESULTS: Minimal to small positive correlations were observed between milestones and ABSITE percentile scores (r = 0.09-0.25), while moderately large correlations were observed between milestones and percentage correct scores (r = 0.65-0.76). The Medical Knowledge 1 (MK1) milestone significantly predicted ABSITE percentage correct scores, and explained 60% of the variance (adjusted R2 = 0.603). MK1 also significantly predicted ABSITE percentile scores, although weaker in magnitude, with MK1 explaining 20% of the variance (adjusted R2 = 0.197). Postgraduate year level and other milestones were not influential predictors of ABSITE scores. CONCLUSIONS: The mid-year MK1 milestone rating was predictive of ABSITE scores and may serve as a useful marker for Clinical Competency Committees to identify residents who could benefit from additional support to prepare for the ABSITE, although given the small exploratory nature of this study, additional research is still needed.

Main outcomes of an RCT to pilot test reporting and feedback to foster research integrity climates in the VA.

BACKGROUND: Assessing the integrity of research climates and sharing such information with research leaders may support research best practices. We report here results of a pilot trial testing the effectiveness of a reporting and feedback intervention using the Survey of Organizational Research Climate (SOuRCe). METHODS: We randomized 41 Veterans Health Administration (VA) facilities to a phone-based intervention designed to help research leaders understand their survey results (enhanced arm) or to an intervention in which results were simply distributed to research leaders (basic arm). Primary outcomes were (1) whether leaders took action, (2) whether actions taken were consistent with the feedback received, and (3) whether responses differed by receptivity to quality improvement input. RESULTS: Research leaders from 25 of 42 (59%) VA facilities consented to participate in the study intervention and follow-up, of which 14 were at facilities randomized to the enhanced arm. We completed follow-up interviews with 21 of the 25 leaders (88%), 12 from enhanced arm facilities. While not statistically significant, the proportion of leaders reporting taking some action in response to the feedback was twice as high in the enhanced arm than in the basic arm (67% vs. 33%, p = .20). While also not statistically significant, a higher proportion of actions taken among facilities in the enhanced arm were responsive to the survey results than in the basic arm (42% vs. 22%, p = .64). CONCLUSIONS: Enhanced feedback of survey results appears to be a promising intervention that may increase the likelihood of responsive action to improve organizational climates. Due to the small sample size of this pilot study, even large percentage-point differences between study arms are not statistically distinguishable. This hypothesis should be tested in a larger trial.

Nicotine Dependence and Willingness to Quit Smoking in Vascular Surgery Patients.

BACKGROUND: Smoking rates have declined; however, it remains the primary modifiable risk factor for vascular disease. While vascular surgeons often advise patients to quit, few provide assistance. We sought to understand patients' interest in quitting and determine factors that influence this willingness to quit. METHODS: Anonymous surveys were given to vascular surgery clinic patients in a single institution over a 5-month period. Demographic information, smoking status, cessation attempts, and barriers to quitting were obtained. Nicotine dependence was determined using the Fagerstrom Test, and willingness to quit was assessed using a contemplation ladder. Patient's willingness to quit was evaluated in relation to symptomatic status, previous/planned operation, and nicotine dependence levels. RESULTS: A total of 490 patients (92%) completed the survey with 109 (22%) current smokers, 195 (40%) former smokers, and 186 (38%) never smokers. Never smokers were more likely to be female and employed while smokers were more likely to be disabled. Although 51% of smokers displayed moderate/high nicotine dependence, 54% demonstrated willingness to consider quitting within 6 months. The primary barrier to cessation identified was previous failed attempt(s) to quit in 44%. Most (90%) had previously attempted quitting, 63% attempted 3 or more times, and the most common technique used was "quitting cold turkey". Fifty-nine percent of patients reported physicians' had offered assistance in cessation, but only 2% had been in a cessation program. There was no correlation between willingness to consider quitting and symptomatic status, previous/planned operation, or nicotine dependence. Smokers of less than 10 cigarettes/day had a lower nicotine dependence level (P = 0.0001) and higher willingness to consider quitting (P = 0.0015), as did those who had fewer failed prior attempts to quit and who did not believe it was too late to quit. CONCLUSIONS: Most of our vascular patients self-report as nonsmokers. Over half of those who smoke demonstrate willingness to consider quitting within 6 months, which is not dependent on nicotine dependence, previous/planned operation, or symptomatic status. Those who smoke less than 10 cigarettes per day, have fewer past failed attempts to quit, and believe that it is not too late to quit are more likely to consider quitting. Vascular surgeons should be more aware of previous quit attempts and patterns of smoking and be proactive in assisting patients in cessation attempts.

Relationships between study habits, burnout, and general surgery resident performance on the American Board of Surgery In-Training Examination.

BACKGROUND: The American Board of Surgery In-Training Examination (ABSITE) is used by programs to evaluate the knowledge and readiness of trainees to sit for the general surgery qualifying examination. It is often used as a tool for resident promotion and may be used by fellowship programs to evaluate candidates. Burnout has been associated with job performance and satisfaction; however, its presence and effects on surgical trainees' performance are not well studied. We sought to understand factors including burnout and study habits that may contribute to performance on the ABSITE examination. METHODS: Anonymous electronic surveys were distributed to all residents at 10 surgical residency programs (n = 326). Questions included demographics as well as study habits, career interests, residency characteristics, and burnout scores using the Oldenburg Burnout Inventory, which assesses burnout because of both exhaustion and disengagement. These surveys were then linked to the individual's 2016 ABSITE and United States Medical Licensing Examination (USMLE) step 1 and 2 scores provided by the programs to determine factors associated with successful ABSITE performance. RESULTS: In total, 48% (n = 157) of the residents completed the survey. Of those completing the survey, 48 (31%) scored in the highest ABSITE quartile (≥75th percentile) and 109 (69%) scored less than the 75th percentile. In univariate analyses, those in the highest ABSITE quartile had significantly higher USMLE step 1 and step 2 scores (P < 0.001), significantly lower burnout scores (disengagement, P < 0.01; exhaustion, P < 0.04), and held opinions that the ABSITE was important for improving their surgical knowledge (P < 0.01). They also read more frequently to prepare for the ABSITE (P < 0.001), had more disciplined study habits (P < 0.001), were more likely to study at the hospital or other public settings (e.g., library, coffee shop compared with at home; P < 0.04), and used active rather than passive study strategies (P < 0.04). Gender, marital status, having children, and debt burden had no correlation with examination success. Backward stepwise multiple regression analysis identified the following independent predictors of ABSITE scores: study location (P < 0.0001), frequency of reading (P = 0.0001), Oldenburg Burnout Inventory exhaustion (P = 0.02), and USMLE step 1 and 2 scores (P = 0.007 and 0.0001, respectively). CONCLUSIONS: Residents who perform higher on the ABSITE have a regular study schedule throughout the year, report less burnout because of exhaustion, study away from home, and have shown success in prior standardized tests. Further study is needed to determine the effects of burnout on clinical duties, career advancement, and satisfaction.

Effects of Mentorship on Graduating Vascular Surgery Trainees.

BACKGROUND: Mentorship within surgery, particularly vascular surgery, has not been extensively evaluated. This study sought to examine mentorship experiences in graduating vascular trainees (VTs). METHODS: An anonymous electronic survey examining current mentor relationships, ideal characteristics, academic productivity, and operative comfort level was emailed to all US graduating 2015 (n = 141) and 2016 (n = 144) VTs during their last year of training. A parallel survey was emailed to program/associate program directors (PDs) of all US vascular training programs (n = 169). RESULTS: Surveys were completed by 65 (38%) PDs and 62 (22%) VTs. Forty-three (69%) VTs identified a mentor with no difference in frequency of having a mentor noted among gender, type of training (fellowship/residency), or geographical region. The majority (41/43, 95%) recognized a vascular surgeon as their mentor; the mentor was assigned to 10 of 43 (23%) trainees and was their PD to 9 of 43 (21%). Ideal mentor characteristics, identified by VTs using 4-point Likert scales, were approachability, supportiveness, and accessibility (3.94, 3.76, and 3.74, respectively), while those of a good mentee were enthusiasm, ethical work, and commitment (3.79, 3.69, and 3.45, respectively). Trainees scored accessibility, approachability, compassion, and same gender significantly higher than PDs as characteristics of ideal mentors (P < 0.05). The majority of PDs thought mentorship increased access to research opportunities, professional networking, and job opportunities, but not operative skill (90%, 95%, 65%, and 18%, respectively). There was no difference between trainees with mentors and those without in terms of reported academic productivity or operative comfort level. Vascular fellows reported higher comfort than residents in open abdominal aortic aneurysm repair (3.32 vs. 2.73, P = 0.006), juxtarenal aneurysm repair (2.8 vs. 2.1, P = 0.02), and aorto-mesenteric bypass (2.57 vs. 1.93, P = 0.03), and neither group was comfortable performing carotid stenting or fenestrated endovascular aneurysm repairs. CONCLUSIONS: There is no difference in academic productivity or case comfort level between surgical trainees with mentors and those without. Possible benefits identified by PDs and trainees include professional networking and increased research and job opportunities. Mentors should be approachable, accessible, supportive, and ethical, while mentees should be enthusiastic, ethical, committed, and approachable. Vascular fellows feel more comfortable with open vascular operations than residents, and nether group feels comfortable with complex endovascular cases. More research is warranted.

Crystal clear or tin ear: how do medical students interpret derogatory comments about patients and other professionals?

PURPOSE: To assess the learning environment at our medical school, third-year medical students complete an 11-item survey called the Learning Environment for Professionalism (LEP) at the end of each clerkship. The LEP survey asks about the frequency of faculty and resident professional and unprofessional behaviors that students observed; two of the items specifically address derogatory comments. This study used focus group methodology to explore how medical students interpret the derogatory comments they reported on the LEP survey. METHODS: Seven focus groups were conducted with 82 medical students after they completed the LEP survey. Analysis of focus group transcripts was performed to better understand the nature and meaning that students ascribe to derogatory comments. RESULTS: The study results provide insights into the types of derogatory comments that medical students heard during their clerkship rotations, why the comments were made and how they were interpreted. Emergent themes, labeled by the authors as 1) 'onstage-offstage', 2) 'one bad apple', and 3) 'pressure cooker environment', highlight the contextual aspects and understandings ascribed by students to the derogatory comments. Incidentally, students felt that the comments were not associated with fatigue, but were associated with cumulative stress and burn-out. CONCLUSIONS: The results suggest students have a clear understanding of the nature of unprofessional comments made by role models during clerkships and point to important systems-related issues that could be leveraged to improve clinical learning environments.

Initial Results from the Survey of Organizational Research Climates (SOuRCe) in the U.S. Department of Veterans Affairs Healthcare System.

BACKGROUND: In service to its core mission of improving the health and well-being of veterans, Veterans Affairs (VA) leadership is committed to supporting research best practices in the VA. Recognizing that the behavior of researchers is influenced by the organizational climates in which they work, efforts to assess the integrity of research climates and share such information with research leadership in VA may be one way to support research best practices. The Survey of Organizational Research Climate (SOuRCe) is the first validated survey instrument specifically designed to assess the organizational climate of research integrity in academic research organizations. The current study reports on an initiative to use the SOuRCe in VA facilities to characterize the organizational research climates and pilot test the effectiveness of using SOuRCe data as a reporting and feedback intervention tool. METHODS: We administered the SOuRCe using a cross-sectional, online survey, with mailed follow-up to non-responders, of research-engaged employees in the research services of a random selection of 42 VA facilities (e.g., Hospitals/Stations) believed to employ 20 or more research staff. We attained a 51% participation rate, yielding more than 5,200 usable surveys. RESULTS: We found a general consistency in organizational research climates across a variety of sub-groups in this random sample of research services in the VA. We also observed similar SOuRCe scale score means, relative rankings of these scales and their internal reliability, in this VA-based sample as we have previously documented in more traditional academic research settings. Results also showed more substantial variability in research climate scores within than between facilities in the VA research service as reflected in meaningful subgroup differences. These findings suggest that the SOuRCe is suitable as an instrument for assessing the research integrity climates in VA and that the tool has similar patterns of results that have been observed in more traditional academic research settings. CONCLUSIONS: The local and specific nature of organizational climates in VA research services, as reflected in variability across sub-groups within individual facilities, has important policy implications. Global, "one-size-fits-all" type initiatives are not likely to yield as much benefit as efforts targeted to specific organizational units or sub-groups and tailored to the specific strengths and weaknesses documented in those locations.

Mobile spaced education for surgery rotation improves National Board of Medical Examiners scores.

BACKGROUND: Use of mobile devices for medical education is increasing. The aim of this study was to describe the implementation of a spaced education-based app study program in a third year medical school surgery rotation and examine its effects on National Board of Medical Examiners (NBME) examination performance. MATERIALS AND METHODS: Case-based questions were created and distributed to students on a voluntary basis via a mobile app. Questions were repeated in a spaced based manner until mastered. Students completed surveys regarding study habits and app use. NBME examination scores and prior academic measures were obtained. Users of the app were compared to non-users of the app and the previous years' class. RESULTS: One hundred fifty-two students were offered the app; 121 (80%) used it. App users had a final NBME score of 77.5% compared with 68.8% (P < 0.01) in non-users, although their prior academic achievement was better. Categorizing them by their academic achievement and intensity of app use demonstrated significantly higher scores in regular users of the app as compared with casual and non-users in mid (78 versus 75 versus 71.2%, P < 0.01) and low-achieving students (71.4 versus 70 versus 64.6, P < 0.01), but not high-achieving students (85.3 versus 81.1 versus 79.7, P = 0.09). Stepwise linear regression modeling revealed intensity of app use accounted for 6% of the variance in NBME scores. CONCLUSIONS: Use of the app-based program was an effective tool associated with higher scores on standardized tests in lower level achieving students.