RESUMO
OBJECTIVE: The objective of this study is to explore the patient characteristics and practice patterns of non-certified surgeons who treat Medicare patients in the United States. SUMMARY BACKGROUND DATA: While most surgeons in the United States are board-certified, non-certified surgeons are permitted to practice in many locations. At the same time, surgical workforce shortages threaten access to surgical care for many patients. It is possible that non-certified surgeons may be able to help fill these access gaps. However, little is known about the practice patterns of non-certified surgeons. METHODS: A 100% sample of Medicare claims data from 2014-2019 were used to identify practicing general surgeons. Surgeons were categorized as certified or non-certified in general surgeryââ based on data from the American Board of Surgery. Surgeon practice patterns and patient characteristics were analyzed. RESULTS: A total of 2,097,206 patient cases were included in the study. These patients were treated by 16,076 surgeons, of which 6% were identified as non-certified surgeons. Compared to certified surgeons, non-certified surgeons were less frequently fellowship-trained (20.5% vs. 24.2%, P=0.008) and more likely to be a foreign medical graduate (14.5% vs. 9.2%, P<0.001). Non-certified surgeons were more frequently practicing in for-profit hospitals (21.2% vs. 14.2%, P<0.001) and critical access hospitals (2.2% vs. 1.3%, P<0.001), and were less likely to practice in a teaching hospital (63.2% vs. 72.4%, P<0.001). Compared to certified surgeons, non-certified surgeons treated more non-White patients (19.6% vs. 14%, P<0.001) as well as a higher percentage of patients in the two lowest socioeconomic status (SES) quintiles (36.2% vs. 29.2%, P<0.001). Operations related to emergency admissions were more common amongst non-certified surgeons (68.8% vs. 55.7%, P<0.001). There were no differences in gender or age of the patients treated by certified and non-certified surgeons. CONCLUSION: For Medicare patients, non-certified surgeons treated more patients who are non-White, of lower SES, and in more rural, critical-access hospitals.
RESUMO
OBJECTIVE: To evaluate severe complications and mortality over years of independent practice among general surgeons. BACKGROUND: Despite concerns that newly graduated general surgeons may be unprepared for independent practice, it is unclear whether patient outcomes differ between early and later career surgeons. METHODS: We used Medicare claims for patients discharged between July 1, 2007 and December 31, 2019 to evaluate 30-day severe complications and mortality for 26 operations defined as core procedures by the American Board of Surgery. Generalized additive mixed models were used to assess the association between surgeon years in practice and 30-day outcomes while adjusting for differences in patient, hospital, and surgeon characteristics. RESULTS: The cohort included 1,329,358 operations performed by 14,399 surgeons. In generalized mixed models, the relative risk (RR) of mortality was higher among surgeons in their first year of practice compared with surgeons in their 15th year of practice [5.5% (95% CI: 4.1%-7.3%) vs 4.7% (95% CI: 3.5%-6.3%), RR: 1.17 (95% CI: 1.11-1.22)]. Similarly, the RR of severe complications was higher among surgeons in their first year of practice compared with surgeons in their 15th year of practice [7.5% (95% CI: 6.6%-8.5%) versus 6.9% (95% CI: 6.1%-7.9%), RR: 1.08 (95% CI: 1.03-1.14)]. When stratified by individual operation, 21 operations had a significantly higher RR of mortality and all 26 operations had a significantly higher RR of severe complications in the first compared with the 15th year of practice. CONCLUSIONS: Among general surgeons performing common operations, rates of mortality and severe complications were higher among newly graduated surgeons compared with later career surgeons.
Assuntos
Medicare , Cirurgiões , Humanos , Estados Unidos/epidemiologia , Idoso , Hospitais , Mortalidade Hospitalar , Competência Clínica , Complicações Pós-Operatórias/epidemiologia , Estudos RetrospectivosRESUMO
INTRODUCTION: Surgery residents who complete a nondesignated preliminary year have an additional year of training compared to those who begin as categorical residents. While this additional year is sometimes perceived negatively, these more experienced residents may outperform traditional categorical (TC) residents in their first year of training. METHODS: Operative assessment ratings were recorded for first year categorical general surgery residents in the United States between 2015 and 2023 using the Society for Improving Medical and Professional Learning assessment platform. Residents were categorized based on the completion of a nondesignated preliminary year ("Previous Prelim" [PP]) or not ("Traditional Categorical"). Ratings were analyzed using generalized mixed effects models. Performance and autonomy outcomes were dichotomized: "less experience" or "more experience" and "no autonomy" or "some autonomy", respectively. Fixed effects included academic month and case complexity, while random effects included resident, faculty, program, and procedure. RESULTS: A total of 34,353 evaluations from 86 general surgery programs were collected. Of these, 829 were evaluations from PP residents. Faculty ratings of PP versus TC revealed no differences in adjusted probabilities of achieving a "more experience" rating (82% versus 76%, P = 0.26) but a higher adjusted probability of achieving a "some autonomy" rating (88% versus 80%, P = 0.04) for PP compared to TC. Analysis of resident self-reported ratings revealed higher adjusted probabilities of a "more experience" rating (77% versus 50%, P = 0.01) and "some autonomy" rating (87% versus 73%; P = 0.02) for PP compared to TC. CONCLUSIONS: First year general surgery residents who previously completed a preliminary year have similar operative performance faculty ratings when compared to their peers.
RESUMO
OBJECTIVE: To examine the alignment between graduating surgical trainee operative performance and a prior survey of surgical program director expectations. BACKGROUND: Surgical trainee operative training is expected to prepare residents to independently perform clinically important surgical procedures. METHODS: We conducted a cross-sectional observational study of US general surgery residents' rated operative performance for Core general surgery procedures. Residents' expected performance on those procedures at the time of graduation was compared to the current list of Core general surgery procedures ranked by their importance for clinical practice, as assessed via a previous national survey of general surgery program directors. We also examined the frequency of individual procedures logged by residents over the course of their training. RESULTS: Operative performance ratings for 29,885 procedures performed by 1861 surgical residents in 54 general surgery programs were analyzed. For each Core general surgery procedure, adjusted mean probability of a graduating resident being deemed practice-ready ranged from 0.59 to 0.99 (mean 0.90, standard deviation 0.08). There was weak correlation between the readiness of trainees to independently perform a procedure at the time of graduation and that procedure's historical importance to clinical practice ( p = 0.22, 95% confidence interval 0.01-0.41, P = 0.06). Residents also continue to have limited opportunities to learn many procedures that are important for clinical practice. CONCLUSION: The operative performance of graduating general surgery residents may not be well aligned with surgical program director expectations.
Assuntos
Cirurgia Geral , Internato e Residência , Humanos , Competência Clínica , Estudos Transversais , Motivação , Inquéritos e Questionários , Cirurgia Geral/educação , Educação de Pós-Graduação em MedicinaRESUMO
ABSTRACT: Medical training programs and health care systems collect ever-increasing amounts of educational and clinical data. These data are collected with the primary purpose of supporting either trainee learning or patient care. Well-established principles guide the secondary use of these data for program evaluation and quality improvement initiatives. More recently, however, these clinical and educational data are also increasingly being used to train artificial intelligence (AI) models. The implications of this relatively unique secondary use of data have not been well explored. These models can support the development of sophisticated AI products that can be commercialized. While these products have the potential to support and improve the educational system, there are challenges related to validity, patient and learner consent, and biased or discriminatory outputs. The authors consider the implications of developing AI models and products using educational and clinical data from learners, discuss the uses of these products within medical education, and outline considerations that should guide the appropriate use of data for this purpose. These issues are further explored by examining how they have been navigated in an educational collaborative.
Assuntos
Inteligência Artificial , Educação Médica , Humanos , Escolaridade , Aprendizagem , Avaliação de Programas e Projetos de SaúdeRESUMO
Introduction: When performance data are provided as feedback to healthcare professionals, they may use it to significantly improve care quality. However, the question of how to provide effective feedback remains unanswered, as decades of evidence have produced a consistent pattern of effects-with wide variation. From a coaching perspective, feedback is often based on a learner's objectives and goals. Furthermore, when coaches provide feedback, it is ideally informed by their understanding of the learner's needs and motivation. We anticipate that a "coaching"-informed approach to feedback may improve its effectiveness in two ways. First, by aligning feedback with healthcare professionals' chosen goals and objectives, and second, by enabling large-scale feedback systems to use new types of data to learn what kind of performance information is motivating in general. Our objective is to propose a conceptual model of precision feedback to support these anticipated enhancements to feedback interventions. Methods: We iteratively represented models of feedback's influence from theories of motivation and behavior change, visualization, and human-computer interaction. Through cycles of discussion and reflection, application to clinical examples, and software development, we implemented and refined the models in a software application to generate precision feedback messages from performance data for anesthesia providers. Results: We propose that precision feedback is feedback that is prioritized according to its motivational potential for a specific recipient. We identified three factors that influence motivational potential: (1) the motivating information in a recipient's performance data, (2) the surprisingness of the motivating information, and (3) a recipient's preferences for motivating information and its visual display. Conclusions: We propose a model of precision feedback that is aligned with leading theories of feedback interventions to support learning about the success of feedback interventions. We plan to evaluate this model in a randomized controlled trial of a precision feedback system that enhances feedback emails to anesthesia providers.
RESUMO
OBJECTIVE: To examine the readiness of general surgery residents in their final year of training to perform 5 common surgical procedures based on their documented performance during training. DESIGN: Intraoperative performance ratings were analyzed using a Bayesian mixed effects approach, adjusting for rater, trainee, procedure, case complexity, and postgraduate year (PGY) as random effects as well as month in academic year and cumulative, procedure-specific performance per trainee as fixed effects. This model was then used to estimate each PGY 5 trainee's final probability of being able to independently perform each procedure. The actual, documented competency rates for individual trainees were then identified across each of the 5 most common general surgery procedures: appendectomy, cholecystectomy, ventral hernia repair, groin hernia repair, and partial colectomy. SETTING: This study was conducted using data from members of the SIMPL collaborative. PARTICIPANTS: A total of 17,248 evaluations of 927 PGY5 general surgery residents were analyzed from 2015 to 2021. RESULTS: The percentage of residents who requested a SIMPL rating during their PGY5 year and achieved a ≥90% probability of being rated as independent, or "Practice-Ready," was 97.4% for appendectomy, 82.4% for cholecystectomy, 43.5% for ventral hernia repair, 24% for groin hernia repair, and 5.3% for partial colectomy. CONCLUSIONS: There is substantial variation in the demonstrated competency of general surgery residents to perform several common surgical procedures at the end of their training. This variation in readiness calls for careful study of how surgical residents can become more adequately prepared to enter independent practice.
Assuntos
Cirurgia Geral , Hérnia Inguinal , Hérnia Ventral , Internato e Residência , Humanos , Teorema de Bayes , Competência Clínica , Educação de Pós-Graduação em Medicina/métodos , Hérnia Inguinal/cirurgia , Hérnia Ventral/cirurgia , Cirurgia Geral/educaçãoRESUMO
Competency-based medical education (CBME) is the future of medical education and relies heavily on high quality assessment. However, the current assessment practices employed by many general surgery graduate medical education training programs are subpar. Assessments often lack reliability and validity evidence, have low faculty engagement, and differ from program to program. Given the importance of assessment in CBME, it is critical that we build a better assessment system for measuring trainee competency. We propose that an ideal system of assessment is standardized, evidence-based, comprehensive, integrated, and continuously improving. In this article, we explore these characteristics and propose next steps to achieve such a system of assessment in general surgery.
Assuntos
Educação de Pós-Graduação em Medicina , Educação Médica , Humanos , Reprodutibilidade dos Testes , Educação Baseada em Competências , Docentes de Medicina , Competência ClínicaRESUMO
OBJECTIVE: Workplace-based assessments (WBAs) play an important role in the assessment of surgical trainees. Because these assessment tools are utilized by a multitude of faculty, inter-rater reliability is important to consider when interpreting WBA data. Although there is evidence supporting the validity of many of these tools, inter-reliability evidence is lacking. This study aimed to evaluate the inter-rater reliability of multiple operative WBA tools utilized in general surgery residency. DESIGN: General surgery residents and teaching faculty were recorded during 6 general surgery operations. Nine faculty raters each reviewed 6 videos and rated each resident on performance (using the Society for Improving Medical Professional Learning, or SIMPL, Performance Scale as well as the operative performance rating system (OPRS) Scale), entrustment (using the ten Cate Entrustment-Supervision Scale), and autonomy (using the Zwisch Scale). The ratings were reviewed for inter-rater reliability using percent agreement and intraclass correlations. PARTICIPANTS: Nine faculty members viewed the videos and assigned ratings for multiple WBAs. RESULTS: Absolute intraclass correlation coefficients for each scale ranged from 0.33 to 0.47. CONCLUSIONS: All single-item WBA scales had low to moderate inter-rater reliability. While rater training may improve inter-rater reliability for single observations, many observations by many raters are needed to reliably assess trainee performance in the workplace.
Assuntos
Competência Clínica , Avaliação Educacional , Cirurgia Geral , Internato e Residência , Local de Trabalho , Cirurgia Geral/educação , Reprodutibilidade dos Testes , Humanos , Avaliação Educacional/métodos , Educação de Pós-Graduação em Medicina/métodos , Gravação em Vídeo , Docentes de Medicina , Masculino , FemininoRESUMO
Importance: Understanding how to translate workplace-based assessment (WBA) ratings into metrics that communicate the ability of a surgeon to perform a procedure would represent a critical advancement in graduate medical education. Objective: To evaluate the association between past and future performance in a comprehensive assessment system for the purpose of assessing point-in-time competence among general surgery trainees. Design, Setting, and Participants: This case series included WBA ratings from September 2015 to September 2021 from the WBA system of the Society for Improving Medical Professional Learning (SIMPL) for all general surgery residents who were provided a rating following an operative performance across 70 programs in the US. The study included ratings for 2605 trainees from 1884 attending surgeon raters. Analyses were conducted between September 2021 and December 2021 using bayesian generalized linear mixed-effects models and marginal predicted probabilities. Exposures: Longitudinal SIMPL ratings. Main Outcomes and Measures: Performance expectations for 193 unique general surgery procedures based on an individual trainee's prior successful ratings for a procedure, clinical year of training, and month of the academic year. Results: Using 63â¯248 SIMPL ratings, the association between prior and future performance was positive (ß, 0.13; 95% credible interval [CrI], 0.12-0.15). The largest source of variation was postgraduate year (α, 3.15; 95% CrI, 1.66-6.03), with rater (α, 1.69; 95% CrI, 1.60-1.78), procedure (α, 1.35; 95% CrI, 1.22-1.51), case complexity (α, 1.30; 95% CrI, 0.42-3.66), and trainee (α, 0.99; 95% CrI, 0.94-1.04) accounting for significant variation in practice ready ratings. After marginalizing overcomplexity and trainee and holding rater constant, mean predicted probabilities had strong overall discrimination (area under the receiver operating characteristic curve, 0.81) and were well calibrated. Conclusions and Relevance: In this study, prior performance was associated with future performance. This association, combined with an overall modeling strategy that accounted for various facets of an assessment task, may offer a strategy for quantifying competence as performance expectations.
Assuntos
Internato e Residência , Humanos , Teorema de Bayes , Motivação , Avaliação Educacional/métodos , Competência Clínica , Educação de Pós-Graduação em MedicinaRESUMO
OBJECTIVE: Competency-based surgical education requires practical assessments and meaningful benchmarks. In otolaryngology, key indicator procedure (KIP) minima are indicators of surgical exposure during training, yet it remains unknown how many times trainees must be evaluated on KIPs to ensure operative competence. Herein, we used Bayesian mixed effects models to compute predicted performance expectations for KIPs. METHODS: From November 2017 to September 2021, a smartphone application (SIMPL OR) was used by attendings at five otolaryngology training programs to rate resident operative performance after each case on a five-level scale. Bayesian mixed effects models were used to estimate the probability that postgraduate year (PGY) 3, 4, or 5 trainees would earn a "practice-ready" (PR) rating on a subsequent evaluation based on their previously earned PR ratings for each KIP. Probabilities of earning a subsequent PR rating were examined for interpretability, and cross-validation was used to assess predictive validity. RESULTS: A total of 842 assessments of KIPs were submitted by 72 attendings for 92 residents PGY 2-5. The predictive model had an average Area Under the Receiver Operating Curve of 0.77. The number of prior PR ratings that senior residents needed to attain a 95% probability of earning a PR rating on a subsequent evaluation was estimated for each KIP. For example, for mastoidectomies, PGY4 residents needed to earn 10 PR ratings whereas PGY5 residents needed 4 PR ratings on average to have a 95% probability of attaining a PR rating on a subsequent evaluation. CONCLUSION: Predictive modeling can inform assessment benchmarks for competency-based surgical education. LEVEL OF EVIDENCE: NA Laryngoscope, 133:3341-3345, 2023.
Assuntos
Cirurgia Geral , Internato e Residência , Otolaringologia , Humanos , Teorema de Bayes , Competência Clínica , Educação de Pós-Graduação em Medicina/métodos , Avaliação Educacional/métodos , Otolaringologia/educação , Cirurgia Geral/educaçãoRESUMO
OBJECTIVE: Assessing surgical trainee operative performance is time- and resource-intensive. To maximize the utility of each assessment, it is important to understand which assessment activities provide the most information about a trainee's performance. The objective of this study is to identify the procedures that best differentiate performance for each general surgery postgraduate year (PGY)-level, leading to recommendations for targeted assessment. DESIGN: The Society for Improving Medical Professional Learning (SIMPL) operative performance ratings were modeled using a multilevel Rasch model which identified the highest and lowest performing trainees for each PGY-level. For each procedure within each PGY-level, a procedural performance discrimination index was calculated by subtracting the proportion of "practice-ready" ratings of the lowest performing trainees from the proportion of "practice-ready" ratings of the highest performing trainees. Four-quadrant plots were created using the median procedure volume and median discrimination index for each PGY-level. All procedures within the upper right quadrant were considered "highly differentiating, high volume" procedures. SETTING: This study was conducted across 70 general surgical residency programs who are members of the SIMPL collaborative. PARTICIPANTS: A total of 54,790 operative performance evaluations of categorical general surgery trainees were collected between 2015 and 2021. Trainees who had at least 1 procedure in common were included. Procedures with less than 25 evaluations per training year were excluded. RESULTS: The total number of evaluations per procedure ranged from 25 to 2,131. Discrimination values were generated for 51 (PGY1), 54 (PGY2), 92 (PGY3), 105 (PGY4), and 103 (PGY5) procedures. Using the above criteria, a total of 12 (PGY1), 15 (PGY2), 22 (PGY3), 21 (PGY4), and 28 (PGY5) procedures were identified as highly differentiating, high volume procedures. CONCLUSIONS: Our study draws on national data to identify procedures which are most useful in differentiating trainee operative performance at each PGY-level. This list of procedures can be used to guide targeted assessment and improve assessment efficiency.
Assuntos
Cirurgia Geral , Internato e Residência , Humanos , Educação de Pós-Graduação em Medicina/métodos , Competência Clínica , Avaliação Educacional/métodos , Cirurgia Geral/educaçãoRESUMO
OBJECTIVE: Feedback is critical for learning, however, gender differences exist in the quality of feedback that trainees receive. For example, narrative feedback on surgical trainees' end-of-block rotations differs based on trainee-faculty gender dyads, with female faculty giving higher quality feedback and male trainees receiving higher quality feedback. Though this represents evidence of gender bias in global evaluations, there is limited understanding of how much bias might be present in operative workplace-based assessments (WBAs). In this study, we explore the quality of narrative feedback among trainee-faculty gender dyads in an operative WBA. DESIGN: A previously validated natural language processing model was used to examine instances of narrative feedback and assign a probability of being characterized as high quality feedback (defined as feedback which was relevant as well as corrective and/or specific). A linear mixed model was performed, with probability of high quality feedback as the outcome, and resident gender, faculty gender, PGY, case complexity, autonomy rating, and operative performance rating as explanatory variables. PARTICIPANTS: Analyses included 67,434 SIMPL operative performance evaluations (2,319 general surgery residents, 70 institutions) collected from September 2015 through September 2021. RESULTS: Of 36.3% evaluations included narrative feedback. Male faculty were more likely to provide narrative feedback compared to female faculty. Mean probabilities of receiving high quality feedback ranged from 81.6 (female faculty-male resident) to 84.7 (male faculty-female resident). Model-based results demonstrated that female residents were more likely to receive high quality feedback (p < 0.01), however, there was no significant difference in probability of high quality narrative feedback based on faculty-resident gender dyad (pâ¯=â¯0.77). CONCLUSIONS: Our study revealed resident gender differences in the probability of receiving high-quality narrative feedback following a general surgery operation. However, we found no significant differences based on faculty-resident gender dyad. Male faculty were more likely to provide narrative feedback compared to their female colleagues. Further research using general surgery resident-specific feedback quality models may be warranted.
Assuntos
Cirurgia Geral , Internato e Residência , Humanos , Masculino , Feminino , Retroalimentação , Competência Clínica , Sexismo , Educação de Pós-Graduação em Medicina/métodos , Cirurgia Geral/educaçãoRESUMO
OBJECTIVE: To develop and validate a model to predict whether patients undergoing ureteroscopy (URS) will receive a stent. METHODS: Using registry data obtained from the Michigan Urological Surgery Improvement Collaborative Reducing Operative Complications from Kidney Stones initiative, we identified patients undergoing URS from 2016 to 2020. We used patients' age, sex, body mass index, size and location of the largest stone, current stent in place, history of any kidney stone procedure, procedure type, and acuity to fit a multivariable logistic regression model to a derivation cohort consisting of a random two-thirds of episodes. Model discrimination and calibration were evaluated in the validation cohort. A sensitivity analysis examined urologist variation using generalized mixed-effect models. RESULTS: We identified 15,048 URS procedures, of which 11,471 (76%) had ureteral stents placed. Older age, male sex, larger stone size, the largest stone being in the ureteropelvic junction, no prior stone surgery, no stent in place, a planned procedure type of laser lithotripsy, and urgent procedure were associated with a higher risk of stent placement. The model achieved an area under the receiver operating characteristic curve of 0.69 (95% CI 0.67, 0.71). Incorporating urologist-level variation improved the area under the receiver operating characteristic curve to 0.83 (95% CI 0.82, 0.84). CONCLUSION: Using a large clinical registry, we developed a multivariable regression model to predict ureteral stent placement following URS. Though well-calibrated, the model had modest discrimination due to heterogeneity in practice patterns in stent placement across urologists.
Assuntos
Cálculos Renais , Litotripsia a Laser , Litotripsia , Ureter , Cálculos Ureterais , Humanos , Masculino , Ureteroscopia/métodos , Cálculos Ureterais/terapia , Cálculos Renais/cirurgia , Ureter/cirurgia , Stents , Resultado do Tratamento , Litotripsia/métodosRESUMO
PURPOSE: Accurate assessment of clinical performance is essential to ensure graduating residents are competent for unsupervised practice. The Accreditation Council for Graduate Medical Education milestones framework is the most widely used competency-based framework in the United States. However, the relationship between residents' milestones competency ratings and their subsequent early career clinical outcomes has not been established. It is important to examine the association between milestones competency ratings of U.S. general surgical residents and those surgeons' patient outcomes in early career practice. METHOD: A retrospective, cross-sectional study was conducted using a sample of national Medicare claims for 23 common, high-risk inpatient general surgical procedures performed between July 1, 2015, and November 30, 2018 (n = 12,400 cases) by nonfellowship-trained U.S. general surgeons. Milestone ratings collected during those surgeons' last year of residency (n = 701 residents) were compared with their risk-adjusted rates of mortality, any complication, or severe complication within 30 days of index operation during their first 2 years of practice. RESULTS: There were no associations between mean milestone competency ratings of graduating general surgery residents and their subsequent early career patient outcomes, including any complication (23% proficient vs 22% not yet proficient; relative risk [RR], 0.97, [95% CI, 0.88-1.08]); severe complication (9% vs 9%, respectively; RR, 1.01, [95% CI, 0.86-1.19]); and mortality (5% vs 5%; RR, 1.07, [95% CI, 0.88-1.30]). Secondary analyses yielded no associations between patient outcomes and milestone ratings specific to technical performance, or between patient outcomes and composites of operative performance, professionalism, or leadership milestones ratings ( P ranged .32-.97). CONCLUSIONS: Milestone ratings of graduating general surgery residents were not associated with the patient outcomes of those surgeons when they performed common, higher-risk procedures in a Medicare population. Efforts to improve how milestones ratings are generated might strengthen their association with early career outcomes.
Assuntos
Internato e Residência , Idoso , Humanos , Estados Unidos , Estudos Retrospectivos , Estudos Transversais , Competência Clínica , Medicare , Educação de Pós-Graduação em Medicina/métodos , Acreditação , Avaliação Educacional/métodosRESUMO
PURPOSE: Despite ongoing efforts to improve surgical education, surgical residents face gaps in their training. However, it is unknown if differences in the training of surgeons are reflected in the patient outcomes of those surgeons once they enter practice. This study aimed to compare the patient outcomes among new surgeons performing partial colectomy-a common procedure for which training is limited-and cholecystectomy-a common procedure for which training is robust. METHOD: The authors retrospectively analyzed all adult Medicare claims data for patients undergoing inpatient partial colectomy and inpatient cholecystectomy between 2007 and 2018. Generalized additive mixed models were used to investigate the associations between surgeon years in practice and risk-adjusted rates of 30-day serious complications and death for patients undergoing partial colectomy and cholecystectomy. RESULTS: A total of 14,449 surgeons at 4,011 hospitals performed 340,114 partial colectomy and 355,923 cholecystectomy inpatient operations during the study period. Patients undergoing a partial colectomy by a surgeon in their 1st vs 15th year of practice had higher rates of serious complications (5.22% [95% CI, 4.85%-5.60%] vs 4.37% [95% CI, 4.22%-4.52%]; P < .01) and death (3.05% [95% CI, 2.92%-3.17%] vs 2.83% [95% CI, 2.75%-2.91%]; P < .01). Patients undergoing a cholecystectomy by a surgeon in their 1st vs 15th year of practice had similar rates of 30-day serious complications (4.11% vs 3.89%; P = .11) and death (1.71% vs 1.70%; P = .93). CONCLUSIONS: Patients undergoing partial colectomy faced a higher risk of serious complications and death when the operation was performed by a new surgeon compared to an experienced surgeon. Conversely, patient outcomes following cholecystectomy were similar for new and experienced surgeons. More attention to partial colectomy during residency training may benefit patients.
Assuntos
Medicare , Cirurgiões , Adulto , Humanos , Idoso , Estados Unidos/epidemiologia , Estudos Retrospectivos , Colecistectomia/efeitos adversos , Colectomia/efeitos adversos , Colectomia/educação , Colectomia/métodosRESUMO
OBJECTIVE: While feedback is an essential component of resident education, there are few large-scale studies examining when and under what conditions formative feedback is provided. Workplace-based assessment systems offer an opportunity to identify factors influencing when faculty provides feedback to trainees. Influential factors affecting feedback may provide targets for increasing and improving feedback in resident education. DESIGN: Data on whether dictated feedback was provided were obtained from the Society for Improving Medical Professional Learning (SIMPL) mobile application. We used generalized linear mixed effects models to identify the degree to which faculty members, procedures, surgical case characteristics, and trainee performance were associated with whether narrative feedback was provided using SIMPL. SETTING: This study was conducted using data from members of the SIMPL collaborative. PARTICIPANTS: 67,434 evaluations from 70 general surgery programs were included from 2015 to 2021. Of these, 25,355 evaluations included dictated feedback. RESULTS: Approximately 61% of the variation in whether dictated feedback was provided was attributable to the individual faculty member. Compared to residents who achieved autonomy ratings of "Active Help," residents who achieved ratings of "Supervision Only" (odds ratio (OR)â¯=â¯0.80, 95% confidence interval (CI)â¯=â¯0.72, 0.88) had a lower likelihood of receiving dictated feedback. Residents who achieved ratings of "Intermediate" (ORâ¯=â¯0.81, CIâ¯=â¯0.74, 0.89), "Practice-Ready" (ORâ¯=â¯0.50, CIâ¯=â¯0.45, 0.57), or "Exceptional (ORâ¯=â¯0.64, CIâ¯=â¯0.54, 0.76) showed a lower likelihood of receiving dictated feedback compared to those rated as "Inexperienced." Cases rated as "High" in terms of complexity were associated with an increased likelihood of having dictation (ORâ¯=â¯1.35, CIâ¯=â¯1.26, 1.44). CONCLUSIONS: The largest contributing factor for whether dictated feedback is included in a SIMPL evaluation are factors specific to the attending surgeon. Resident performance, resident autonomy, and case complexity had only modest associations with feedback decisions. Efforts to improve the amount of formative feedback for trainees should be directed towards reducing the variation in which attending surgeons elect to provide feedback.
Assuntos
Cirurgia Geral , Internato e Residência , Humanos , Retroalimentação , Competência Clínica , Local de Trabalho , Feedback Formativo , Cirurgia Geral/educaçãoRESUMO
OBJECTIVE: Workplace-based assessment is increasingly prevalent in surgical education, especially for assessing operative skill. With current implementations, not all observed clinical performances are assessed, in part because trainees often have discretion about when they seek assessment. As a result, these samples of observed operative performances may not be representative of the full breadth of experience of surgical trainees. Therefore, analyses of these samples may be biased. We aimed to benchmark patterns of procedures logged in the SIMPL operative performance assessment system against records of trainee experience in Accreditation Council for Graduate Medical Education (ACGME) case logs. DESIGN: We analyzed SIMPL longitudinal intraoperative performance assessments from categorical trainees in US general surgery residency programs. We compared overall patterns of how procedures are logged in SIMPL and in ACGME case logs using a Pearson correlation, and we examined differences in how individual procedures are logged in each system using Fisher's exact test. RESULTS: Total procedure frequency from the SIMPL dataset was strongly correlated with total procedure frequency from ACGME case logs (râ¯=â¯0.86, 95% CI 0.80-0.90). A subset of these procedures (10 of 116 procedures) was logged more frequently in the SIMPL dataset. These 10 procedures accounted for 56% of SIMPL observations and 30% of ACGME logged cases. Case complexity was comparable for assessments initiated by residents and faculty. CONCLUSIONS: Samples of intraoperative performance ratings gathered using the SIMPL application largely resemble ACGME case logs. There is no evidence to indicate that residents preferentially select fewer complex cases for assessment.
Assuntos
Cirurgia Geral , Internato e Residência , Acreditação , Competência Clínica , Educação de Pós-Graduação em Medicina/métodos , Cirurgia Geral/educação , Local de TrabalhoRESUMO
BACKGROUND: Many US general surgery residents are interested in global surgery, but their competence with key procedures is unknown. METHODS: Using a registry managed by the Society for Improving Medical Professional Learning (SIMPL), we extracted longitudinal operative performance ratings data for a national cohort of US general surgery residents. Operative performance at the time of graduation was estimated via a Bayesian generalized linear mixed model. RESULTS: Operative performance ratings for 12,976 procedures performed by 1584 residents in 52 general surgery programs were analyzed. These spanned 17 of 31 (55%) procedures deemed important for global surgical practice. For these procedures, the probability of a graduating resident being deemed competent to perform a procedure was 0.95 (95% confidence interval 0.86-1.00) but was less than 0.9 for 3 observed procedures. CONCLUSION: Our results highlight gaps in the preparedness of US general surgery trainees to perform procedures deemed most important for global surgery settings.
Assuntos
Cirurgia Geral , Internato e Residência , Teorema de Bayes , Competência Clínica , Estudos de Coortes , Educação de Pós-Graduação em Medicina , Cirurgia Geral/educação , HumanosRESUMO
OBJECTIVE: Surgical trainees are subject to pressure from variety of stakeholders to secure board certification from the American Board of Surgery (ABS). To meet these expectations, trainees must pass a written qualifying exam (QE) and an oral certifying exam (CE) within 7 years of completing general surgery residency. Board certification outcomes for candidates who fail either the QE or CE examination are not well characterized, but this information could help candidates, policymakers, and other stakeholders make informed decisions about how to respond to examination failure. METHODS: We retrospectively examined ABS records for all surgeons who completed general surgery residency from 2000 to 2013 and attempted general surgery board certification. RESULTS: Among 14,483 surgeons who attempted general surgery certification, 13,566 (94%) passed both the QE and CE within the 7-year certification window. Of those who did ultimately obtain certification, 97% passed the QE within 2 attempts and 97% passed the CE within 2 attempts. For those who failed either the QE or the CE twice, 67% ultimately obtained certification. CONCLUSIONS: Most surgeons who obtained ABS general surgery board certification did so within 2 attempts at each board examination. Candidates who fail either examination twice are less likely to achieve board certification.