Characteristics and Practice Patterns of Non-certified Surgeons Treating Medicare Patients.

OBJECTIVE: The objective of this study is to explore the patient characteristics and practice patterns of non-certified surgeons who treat Medicare patients in the United States. SUMMARY BACKGROUND DATA: While most surgeons in the United States are board-certified, non-certified surgeons are permitted to practice in many locations. At the same time, surgical workforce shortages threaten access to surgical care for many patients. It is possible that non-certified surgeons may be able to help fill these access gaps. However, little is known about the practice patterns of non-certified surgeons. METHODS: A 100% sample of Medicare claims data from 2014-2019 were used to identify practicing general surgeons. Surgeons were categorized as certified or non-certified in general surgery​​ based on data from the American Board of Surgery. Surgeon practice patterns and patient characteristics were analyzed. RESULTS: A total of 2,097,206 patient cases were included in the study. These patients were treated by 16,076 surgeons, of which 6% were identified as non-certified surgeons. Compared to certified surgeons, non-certified surgeons were less frequently fellowship-trained (20.5% vs. 24.2%, P=0.008) and more likely to be a foreign medical graduate (14.5% vs. 9.2%, P<0.001). Non-certified surgeons were more frequently practicing in for-profit hospitals (21.2% vs. 14.2%, P<0.001) and critical access hospitals (2.2% vs. 1.3%, P<0.001), and were less likely to practice in a teaching hospital (63.2% vs. 72.4%, P<0.001). Compared to certified surgeons, non-certified surgeons treated more non-White patients (19.6% vs. 14%, P<0.001) as well as a higher percentage of patients in the two lowest socioeconomic status (SES) quintiles (36.2% vs. 29.2%, P<0.001). Operations related to emergency admissions were more common amongst non-certified surgeons (68.8% vs. 55.7%, P<0.001). There were no differences in gender or age of the patients treated by certified and non-certified surgeons. CONCLUSION: For Medicare patients, non-certified surgeons treated more patients who are non-White, of lower SES, and in more rural, critical-access hospitals.

Evaluating Performance and Autonomy Levels of Previous Preliminary Surgery Interns.

INTRODUCTION: Surgery residents who complete a nondesignated preliminary year have an additional year of training compared to those who begin as categorical residents. While this additional year is sometimes perceived negatively, these more experienced residents may outperform traditional categorical (TC) residents in their first year of training. METHODS: Operative assessment ratings were recorded for first year categorical general surgery residents in the United States between 2015 and 2023 using the Society for Improving Medical and Professional Learning assessment platform. Residents were categorized based on the completion of a nondesignated preliminary year ("Previous Prelim" [PP]) or not ("Traditional Categorical"). Ratings were analyzed using generalized mixed effects models. Performance and autonomy outcomes were dichotomized: "less experience" or "more experience" and "no autonomy" or "some autonomy", respectively. Fixed effects included academic month and case complexity, while random effects included resident, faculty, program, and procedure. RESULTS: A total of 34,353 evaluations from 86 general surgery programs were collected. Of these, 829 were evaluations from PP residents. Faculty ratings of PP versus TC revealed no differences in adjusted probabilities of achieving a "more experience" rating (82% versus 76%, P = 0.26) but a higher adjusted probability of achieving a "some autonomy" rating (88% versus 80%, P = 0.04) for PP compared to TC. Analysis of resident self-reported ratings revealed higher adjusted probabilities of a "more experience" rating (77% versus 50%, P = 0.01) and "some autonomy" rating (87% versus 73%; P = 0.02) for PP compared to TC. CONCLUSIONS: First year general surgery residents who previously completed a preliminary year have similar operative performance faculty ratings when compared to their peers.

Simplified cardiovascular index may be the best comorbidity index for clinical use in prediction of mortality for renal cancer patients.

OBJECTIVE: Understanding the relationship between comorbidities and life expectancy is important in cancer patients who carry risks of cancer and noncancer-related mortality. Comorbidity indices (CI) are tools to provide an objective measure of competing risks of death. We sought to determine which CI might be best incorporated into clinical practice for patients with suspected renal cancer. MATERIALS AND METHODS: 1572 patients diagnosed with renal masses (stage I-IV) between 1998 and 2016 were analyzed for this study. Patient data were gathered from a community-based health center. Comorbidities were evaluated individually, and with 1 of 4 CI: Charlson (CCI), updated CCI (uCCI), age-adjusted CCI (aCCI), and simplified cardiovascular index (CVI). Cox-proportional hazard analysis of all-cause mortality was performed using the four CI, adjusting for the 4 CI, adjusting for age, gender, race, tumor size, and tumor stage. RESULTS: Univariable analyses revealed the four CI were significant predictors of mortality (P < 0.05), as were age, gender, tumor size, and stage. Comorbid conditions at diagnosis included hypertension (47.8%), diabetes mellitus (47.2%), coronary artery disease (41.1%), chronic kidney disease (31.8%), peripheral vascular disease (8.0%), congestive heart failure (5.7%), chronic obstructive pulmonary disease (5.7%), and cerebrovascular disease (2.0%). When analyzing the 4 CI in multivariable survival analyses accounting for factors available at diagnosis, and analyses incorporating pathologic and recurrence data, only CVI score and uCCI remained statistically significant (P < 0.05). Limitations of this work are the retrospective nature of data collection and data from a single institution, limiting the generalizability. CONCLUSION: Increasing comorbidity, age, tumor size, and cM stage are predictors of ACM for suspected renal cancer patients. CVI appears to provide comparable information to various iterations of CCI (uCCI, aCCI) while being the simplest to use. Utilization of CVI may assist clinicians and patients when considering between interventional and noninterventional approaches for suspected renal cancer.

Considering the Secondary Use of Clinical and Educational Data to Facilitate the Development of Artificial Intelligence Models.

ABSTRACT: Medical training programs and health care systems collect ever-increasing amounts of educational and clinical data. These data are collected with the primary purpose of supporting either trainee learning or patient care. Well-established principles guide the secondary use of these data for program evaluation and quality improvement initiatives. More recently, however, these clinical and educational data are also increasingly being used to train artificial intelligence (AI) models. The implications of this relatively unique secondary use of data have not been well explored. These models can support the development of sophisticated AI products that can be commercialized. While these products have the potential to support and improve the educational system, there are challenges related to validity, patient and learner consent, and biased or discriminatory outputs. The authors consider the implications of developing AI models and products using educational and clinical data from learners, discuss the uses of these products within medical education, and outline considerations that should guide the appropriate use of data for this purpose. These issues are further explored by examining how they have been navigated in an educational collaborative.

Precision feedback: A conceptual model.

Introduction: When performance data are provided as feedback to healthcare professionals, they may use it to significantly improve care quality. However, the question of how to provide effective feedback remains unanswered, as decades of evidence have produced a consistent pattern of effects-with wide variation. From a coaching perspective, feedback is often based on a learner's objectives and goals. Furthermore, when coaches provide feedback, it is ideally informed by their understanding of the learner's needs and motivation. We anticipate that a "coaching"-informed approach to feedback may improve its effectiveness in two ways. First, by aligning feedback with healthcare professionals' chosen goals and objectives, and second, by enabling large-scale feedback systems to use new types of data to learn what kind of performance information is motivating in general. Our objective is to propose a conceptual model of precision feedback to support these anticipated enhancements to feedback interventions. Methods: We iteratively represented models of feedback's influence from theories of motivation and behavior change, visualization, and human-computer interaction. Through cycles of discussion and reflection, application to clinical examples, and software development, we implemented and refined the models in a software application to generate precision feedback messages from performance data for anesthesia providers. Results: We propose that precision feedback is feedback that is prioritized according to its motivational potential for a specific recipient. We identified three factors that influence motivational potential: (1) the motivating information in a recipient's performance data, (2) the surprisingness of the motivating information, and (3) a recipient's preferences for motivating information and its visual display. Conclusions: We propose a model of precision feedback that is aligned with leading theories of feedback interventions to support learning about the success of feedback interventions. We plan to evaluate this model in a randomized controlled trial of a precision feedback system that enhances feedback emails to anesthesia providers.

An Ideal System of Assessment to Support Competency-Based Graduate Medical Education: Key Attributes and Proposed Next Steps.

Competency-based medical education (CBME) is the future of medical education and relies heavily on high quality assessment. However, the current assessment practices employed by many general surgery graduate medical education training programs are subpar. Assessments often lack reliability and validity evidence, have low faculty engagement, and differ from program to program. Given the importance of assessment in CBME, it is critical that we build a better assessment system for measuring trainee competency. We propose that an ideal system of assessment is standardized, evidence-based, comprehensive, integrated, and continuously improving. In this article, we explore these characteristics and propose next steps to achieve such a system of assessment in general surgery.

Development and validation of a multicenter Cox regression model to predict all-cause mortality in patients with renal masses suspicious for renal cancer.

OBJECTIVE: Life expectancy models are useful tools to support clinical decision-making. Prior models have not been used widely in clinical practice for patients with renal masses. We sought to develop and validate a model to predict life expectancy following the detection of a localized renal mass suspicious for renal cell carcinoma. MATERIALS AND METHODS: Using retrospective data from 2 large centers, we identified patients diagnosed with clinically localized renal parenchymal masses from 1998 to 2018. After 2:1 random sampling into a derivation and validation cohort stratified by site, we used age, sex, log-transformed tumor size, simplified cardiovascular index and planned treatment to fit a Cox regression model to predict all-cause mortality from the time of diagnosis. The model's discrimination was evaluated using a C-statistic, and calibration was evaluated visually at 1, 5, and 10 years. RESULTS: We identified 2,667 patients (1,386 at Corewell Health and 1,281 at Johns Hopkins) with renal masses. Of these, 420 (16%) died with a median follow-up of 5.2 years (interquartile range 2.2-8.3). Statistically significant predictors in the multivariable Cox regression model were age (hazard ratio [HR] 1.04; 95% confidence interval [CI] 1.03-1.05); male sex (HR 1.40; 95% CI 1.08-1.81); log-transformed tumor size (HR 1.71; 95% CI 1.30-2.24); cardiovascular index (HR 1.48; 95% CI 1.32-1.67), and planned treatment (HR: 0.10, 95% CI: 0.06-0.18 for kidney-sparing intervention and HR: 0.20, 95% CI: 0.11-0.35 for radical nephrectomy vs. no intervention). The model achieved a C-statistic of 0.74 in the derivation cohort and 0.73 in the validation cohort. The model was well-calibrated at 1, 5, and 10 years of follow-up. CONCLUSIONS: For patients with localized renal masses, accurate determination of life expectancy is essential for decision-making regarding intervention vs. active surveillance as a primary treatment modality. We have made available a simple tool for this purpose.

How Reliable are Single-Question Workplace-Based Assessments in Surgery?

OBJECTIVE: Workplace-based assessments (WBAs) play an important role in the assessment of surgical trainees. Because these assessment tools are utilized by a multitude of faculty, inter-rater reliability is important to consider when interpreting WBA data. Although there is evidence supporting the validity of many of these tools, inter-reliability evidence is lacking. This study aimed to evaluate the inter-rater reliability of multiple operative WBA tools utilized in general surgery residency. DESIGN: General surgery residents and teaching faculty were recorded during 6 general surgery operations. Nine faculty raters each reviewed 6 videos and rated each resident on performance (using the Society for Improving Medical Professional Learning, or SIMPL, Performance Scale as well as the operative performance rating system (OPRS) Scale), entrustment (using the ten Cate Entrustment-Supervision Scale), and autonomy (using the Zwisch Scale). The ratings were reviewed for inter-rater reliability using percent agreement and intraclass correlations. PARTICIPANTS: Nine faculty members viewed the videos and assigned ratings for multiple WBAs. RESULTS: Absolute intraclass correlation coefficients for each scale ranged from 0.33 to 0.47. CONCLUSIONS: All single-item WBA scales had low to moderate inter-rater reliability. While rater training may improve inter-rater reliability for single observations, many observations by many raters are needed to reliably assess trainee performance in the workplace.