Discrimination, Abuse, Harassment, and Burnout in Surgical Residency Training.

BACKGROUND: Physicians, particularly trainees and those in surgical subspecialties, are at risk for burnout. Mistreatment (i.e., discrimination, verbal or physical abuse, and sexual harassment) may contribute to burnout and suicidal thoughts. METHODS: A cross-sectional national survey of general surgery residents administered with the 2018 American Board of Surgery In-Training Examination assessed mistreatment, burnout (evaluated with the use of the modified Maslach Burnout Inventory), and suicidal thoughts during the past year. We used multivariable logistic-regression models to assess the association of mistreatment with burnout and suicidal thoughts. The survey asked residents to report their gender. RESULTS: Among 7409 residents (99.3% of the eligible residents) from all 262 surgical residency programs, 31.9% reported discrimination based on their self-identified gender, 16.6% reported racial discrimination, 30.3% reported verbal or physical abuse (or both), and 10.3% reported sexual harassment. Rates of all mistreatment measures were higher among women; 65.1% of the women reported gender discrimination and 19.9% reported sexual harassment. Patients and patients' families were the most frequent sources of gender discrimination (as reported by 43.6% of residents) and racial discrimination (47.4%), whereas attending surgeons were the most frequent sources of sexual harassment (27.2%) and abuse (51.9%). Proportion of residents reporting mistreatment varied considerably among residency programs (e.g., ranging from 0 to 66.7% for verbal abuse). Weekly burnout symptoms were reported by 38.5% of residents, and 4.5% reported having had suicidal thoughts during the past year. Residents who reported exposure to discrimination, abuse, or harassment at least a few times per month were more likely than residents with no reported mistreatment exposures to have symptoms of burnout (odds ratio, 2.94; 95% confidence interval [CI], 2.58 to 3.36) and suicidal thoughts (odds ratio, 3.07; 95% CI, 2.25 to 4.19). Although models that were not adjusted for mistreatment showed that women were more likely than men to report burnout symptoms (42.4% vs. 35.9%; odds ratio, 1.33; 95% CI, 1.20 to 1.48), the difference was no longer evident after the models were adjusted for mistreatment (odds ratio, 0.90; 95% CI, 0.80 to 1.00). CONCLUSIONS: Mistreatment occurs frequently among general surgery residents, especially women, and is associated with burnout and suicidal thoughts.

Association of Surgical Resident Wellness With Medical Errors and Patient Outcomes.

OBJECTIVES: The aims of this study were to: (1) measure the prevalence of self-reported medical error among general surgery trainees, (2) assess the association between general surgery resident wellness (ie, burnout and poor psychiatric well-being) and self-reported medical error, and (3) examine the association between program-level wellness and objectively measured patient outcomes. SUMMARY OF BACKGROUND DATA: Poor wellness is prevalent among surgical trainees but the impact on medical error and objective patient outcomes (eg, morbidity or mortality) is unclear as existing studies are limited to physician and patient self-report of events and errors, small cohorts, or examine few outcomes. METHODS: A cross-sectional survey was administered immediately following the January 2017 American Board of Surgery In-training Examination to clinically active general surgery residents to assess resident wellness and self-reported error. Postoperative patient outcomes were ascertained using a validated national clinical data registry. Associations were examined using multivariable logistic regression models. RESULTS: Over a 6-month period, 22.5% of residents reported committing a near miss medical error, and 6.9% reported committing a harmful medical error. Residents were more likely to report a harmful medical error if they reported frequent burnout symptoms [odds ratio 2.71 (95% confidence interval 2.16-3.41)] or poor psychiatric well-being [odds ratio 2.36 (95% confidence interval 1.92-2.90)]. However, there were no significant associations between program-level resident wellness and any of the independently, objectively measured postoperative American College of Surgeons National Surgical Quality improvement Program outcomes examined. CONCLUSIONS: Although surgical residents with poor wellness were more likely to self-report a harmful medical error, there was not a higher rate of objectively reported outcomes for surgical patients treated at hospitals with higher rates of burnout or poor psychiatric well-being.

Post Hepatectomy Liver Failure Risk Calculator for Preoperative and Early Postoperative Period Following Major Hepatectomy.

BACKGROUND: Post hepatectomy liver failure (PHLF) is associated with significant perioperative morbidity and mortality. A tool to identify patients at risk for PHLF may allow for earlier intervention to mitigate its severity and help clinicians when counseling patients. Our objective was to develop a PHLF risk calculator. STUDY DESIGN: Patients who underwent hepatectomy for any indication from 2014 to 2017 were identified from ACS NSQIP. A multivariable logistic regression model was developed that included preoperative and intraoperative variables. Model fit was assessed for discrimination using the C-statistic, and calibration using Hosmer and Lemeshow (HL) Chi square. Validation of the calculator was performed utilizing tenfold cross validation. RESULTS: Among 15,636 hepatectomy patients analyzed, the overall incidence of clinically significant PHLF was 2.8%. Preoperative patient factors associated with increased PHLF were male gender, preoperative ascites within 30 days of surgery, higher ASA class, preoperative total bilirubin greater than 1.2 mg/dl, and AST greater than 40 units/l. Disease related factors associated with PHLF included histology, and use of neoadjuvant therapy. Intraoperative factors associated with PHLF were extent of resection, open surgical approach, abnormal liver texture, and biliary reconstruction. The calculator's C-statistic was 0.83 and the HL Chi square was 10.9 (p = 0.21) demonstrating excellent discrimination and calibration. On tenfold cross validation, the mean test group C-statistic was 0.82 and the HL p value was 0.26. CONCLUSION: We present a multi-institutional preoperative and early postoperative PHLF risk calculator, which demonstrated excellent discrimination and calibration. This tool can be used to help identify high-risk patients to facilitate earlier interventions.

An Empirical National Assessment of the Learning Environment and Factors Associated With Program Culture.

OBJECTIVES: To empirically describe surgical residency program culture and assess program characteristics associated with program culture. SUMMARY BACKGROUND DATA: Despite concerns about the impact of the learning environment on trainees, empirical data have not been available to examine and compare program-level differences in residency culture. METHODS: Following the 2018 American Board of Surgery In-Training Examination, a cross-sectional survey was administered to all US general surgery residents. Survey items were analyzed using principal component analysis to derive composite measures of program culture. Associations between program characteristics and composite measures of culture were assessed. RESULTS: Analysis included 7387 residents at 260 training programs (99.3% response rate). Principal component analysis suggested that program culture may be described by 2 components: Wellness and Negative Exposures. Twenty-six programs (10.0%) were in the worst quartile for both Wellness and Negative Exposure components. These programs had significantly higher rates of duty hour violations (23.3% vs 11.1%), verbal/physical abuse (41.6% vs 28.6%), gender discrimination (78.7% vs 64.5%), sexual harassment (30.8% vs 16.7%), burnout (54.9% vs 35.0%), and thoughts of attrition (21.6% vs 10.8%; all P < 0.001). Being in the worst quartile of both components was associated with percentage of female residents in the program (P = 0.011), but not program location, academic affiliation, size, or faculty demographics. CONCLUSIONS: Residency culture was characterized by poor resident wellness and frequent negative exposures and was generally not associated with structural program characteristics. Additional qualitative and quantitative studies are needed to explore unmeasured local social dynamics that may underlie measured differences in program culture.

National Cluster-Randomized Trial of Duty-Hour Flexibility in Surgical Training.

BACKGROUND: Concerns persist regarding the effect of current surgical resident duty-hour policies on patient outcomes, resident education, and resident well-being. METHODS: We conducted a national, cluster-randomized, pragmatic, noninferiority trial involving 117 general surgery residency programs in the United States (2014-2015 academic year). Programs were randomly assigned to current Accreditation Council for Graduate Medical Education (ACGME) duty-hour policies (standard-policy group) or more flexible policies that waived rules on maximum shift lengths and time off between shifts (flexible-policy group). Outcomes included the 30-day rate of postoperative death or serious complications (primary outcome), other postoperative complications, and resident perceptions and satisfaction regarding their well-being, education, and patient care. RESULTS: In an analysis of data from 138,691 patients, flexible, less-restrictive duty-hour policies were not associated with an increased rate of death or serious complications (9.1% in the flexible-policy group and 9.0% in the standard-policy group, P=0.92; unadjusted odds ratio for the flexible-policy group, 0.96; 92% confidence interval, 0.87 to 1.06; P=0.44; noninferiority criteria satisfied) or of any secondary postoperative outcomes studied. Among 4330 residents, those in programs assigned to flexible policies did not report significantly greater dissatisfaction with overall education quality (11.0% in the flexible-policy group and 10.7% in the standard-policy group, P=0.86) or well-being (14.9% and 12.0%, respectively; P=0.10). Residents under flexible policies were less likely than those under standard policies to perceive negative effects of duty-hour policies on multiple aspects of patient safety, continuity of care, professionalism, and resident education but were more likely to perceive negative effects on personal activities. There were no significant differences between study groups in resident-reported perception of the effect of fatigue on personal or patient safety. Residents in the flexible-policy group were less likely than those in the standard-policy group to report leaving during an operation (7.0% vs. 13.2%, P<0.001) or handing off active patient issues (32.0% vs. 46.3%, P<0.001). CONCLUSIONS: As compared with standard duty-hour policies, flexible, less-restrictive duty-hour policies for surgical residents were associated with noninferior patient outcomes and no significant difference in residents' satisfaction with overall well-being and education quality. (FIRST ClinicalTrials.gov number, NCT02050789.).

Outcomes of Concurrent Operations: Results From the American College of Surgeons' National Surgical Quality Improvement Program.

OBJECTIVE: To determine whether concurrently performed operations are associated with an increased risk for adverse events. BACKGROUND: Concurrent operations occur when a surgeon is simultaneously responsible for critical portions of 2 or more operations. How this practice affects patient outcomes is unknown. METHODS: Using American College of Surgeons' National Surgical Quality Improvement Program data from 2014 to 2015, operations were considered concurrent if they overlapped by ≥60 minutes or in their entirety. Propensity-score-matched cohorts were constructed to compare death or serious morbidity (DSM), unplanned reoperation, and unplanned readmission in concurrent versus non-concurrent operations. Multilevel hierarchical regression was used to account for the clustered nature of the data while controlling for procedure and case mix. RESULTS: There were 1430 (32.3%) surgeons from 390 (77.7%) hospitals who performed 12,010 (2.3%) concurrent operations. Plastic surgery (n = 393 [13.7%]), otolaryngology (n = 470 [11.2%]), and neurosurgery (n = 2067 [8.4%]) were specialties with the highest proportion of concurrent operations. Spine procedures were the most frequent concurrent procedures overall (n = 2059/12,010 [17.1%]). Unadjusted rates of DSM (9.0% vs 7.1%; P < 0.001), reoperation (3.6% vs 2.7%; P < 0.001), and readmission (6.9% vs 5.1%; P < 0.001) were greater in the concurrent operation cohort versus the non-concurrent. After propensity score matching and risk-adjustment, there was no significant association of concurrence with DSM (odds ratio [OR] 1.08; 95% confidence interval [CI] 0.96-1.21), reoperation (OR 1.16; 95% CI 0.96-1.40), or readmission (OR 1.14; 95% CI 0.99-1.29). CONCLUSIONS: In these analyses, concurrent operations were not detected to increase the risk for adverse outcomes. These results do not lessen the need for further studies, continuous self-regulation and proactive disclosure to patients.

Evaluation of the ProPublica Surgeon Scorecard "Adjusted Complication Rate" Measure Specifications.

OBJECTIVES: The ProPublica Surgeon Scorecard is the first nationwide, multispecialty public reporting of individual surgeon outcomes. However, ProPublica's use of a previously undescribed outcome measure (composite of in-hospital mortality or 30-day related readmission) and inclusion of only inpatients have been questioned. Our objectives were to (1) determine the proportion of cases excluded by ProPublica's specifications, (2) assess the proportion of inpatient complications excluded from ProPublica's measure, and (3) examine the validity of ProPublica's outcome measure by comparing performance on the measure to well-established postoperative outcome measures. METHODS: Using ACS-NSQIP data (2012-2014) for 8 ProPublica procedures and for All Operations, the proportion of cases meeting all ProPublica inclusion criteria was determined. We assessed the proportion of complications occurring inpatient, and thus not considered by ProPublica's measure. Finally, we compared risk-adjusted performance based on ProPublica's measure specifications to established ACS-NSQIP outcome measure performance (eg, death/serious morbidity, mortality). RESULTS: ProPublica's inclusion criteria resulted in elimination of 82% of all operations from assessment (range: 42% for total knee arthroplasty to 96% for laparoscopic cholecystectomy). For all ProPublica operations combined, 84% of complications occur during inpatient hospitalization (range: 61% for TURP to 88% for total hip arthroplasty), and are thus missed by the ProPublica measure. Hospital-level performance on the ProPublica measure correlated weakly with established complication measures, but correlated strongly with readmission (R = 0.834, P < 0.001). CONCLUSIONS: ProPublica's outcome measure specifications exclude 82% of cases, miss 84% of postoperative complications, and correlate poorly with well-established postoperative outcomes. Thus, the validity of the ProPublica Surgeon Scorecard is questionable.

Firearm-related injury and death in the United States: a call to action from 8 health professional organizations and the American Bar Association.

Deaths and injuries related to firearms constitute a major public health problem in the United States. In response to firearm violence and other firearm-related injuries and deaths, an interdisciplinary, interprofessional group of leaders of 8 national health professional organizations and the American Bar Association, representing the official policy positions of their organizations, advocate a series of measures aimed at reducing the health and public health consequences of firearms. The specific recommendations include universal background checks of gun purchasers, elimination of physician "gag laws," restricting the manufacture and sale of military-style assault weapons and large-capacity magazines for civilian use, and research to support strategies for reducing firearm-related injuries and deaths. The health professional organizations also advocate for improved access to mental health services and avoidance of stigmatization of persons with mental and substance use disorders through blanket reporting laws. The American Bar Association, acting through its Standing Committee on Gun Violence, confirms that none of these recommendations conflict with the Second Amendment or previous rulings of the U.S. Supreme Court.

The American College of Surgeons National Surgical Quality Improvement Program: achieving better and safer surgery.

BACKGROUND: The American College of Surgeons (ACS) National Surgical Quality Improvement Program (NSQIP), in operation since late 2004, evaluates surgical quality and safety by feeding back valid, timely, risk-adjusted outcomes, which providers use to improve care. METHODS: A number of components have been developed and refined in the more than a decade since ACS NSQIP's initiation. These items can be grouped into areas of data collection, case sampling, risk adjustment, feedback reporting, the expansion into procedure-targeted sampling, development of improvement collaboratives, and the development of improvement tools. Although ACS NSQIP was originally designed as a hospital-based program, it now also allows for surgeon-specific reporting that can be used by individual surgeons as a feedback tool to improve their performance. RESULTS: There are more than 600 ACS NSQIP hospitals in 49 of the 50 states of the United States and in 13 other countries. Virtually all surgical (sub)specialties are touched by ACS NSQIP, which contains several million patient records and more than 100 statistically risk-adjusted models. In studies that have used ACS NSQIP clinical data, demonstrable improvement has been reported in local hospitals, in regional collaboratives, and across the program overall. Concomitantly, substantial cost savings for individual hospitals, as well as at regional and national levels, have been reported. CONCLUSION: ACS NSQIP has not only demonstrated how and why the use of accurate clinical data is crucial, but also how the program, through its risk-adjusted feedback, improvement tools, and hospital collaboratives, helps hospitals and providers to achieve safer surgery and better patient care.

Transfusion of plasma, platelets, and red blood cells in a 1:1:1 vs a 1:1:2 ratio and mortality in patients with severe trauma: the PROPPR randomized clinical trial.

IMPORTANCE: Severely injured patients experiencing hemorrhagic shock often require massive transfusion. Earlier transfusion with higher blood product ratios (plasma, platelets, and red blood cells), defined as damage control resuscitation, has been associated with improved outcomes; however, there have been no large multicenter clinical trials. OBJECTIVE: To determine the effectiveness and safety of transfusing patients with severe trauma and major bleeding using plasma, platelets, and red blood cells in a 1:1:1 ratio compared with a 1:1:2 ratio. DESIGN, SETTING, AND PARTICIPANTS: Pragmatic, phase 3, multisite, randomized clinical trial of 680 severely injured patients who arrived at 1 of 12 level I trauma centers in North America directly from the scene and were predicted to require massive transfusion between August 2012 and December 2013. INTERVENTIONS: Blood product ratios of 1:1:1 (338 patients) vs 1:1:2 (342 patients) during active resuscitation in addition to all local standard-of-care interventions (uncontrolled). MAIN OUTCOMES AND MEASURES: Primary outcomes were 24-hour and 30-day all-cause mortality. Prespecified ancillary outcomes included time to hemostasis, blood product volumes transfused, complications, incidence of surgical procedures, and functional status. RESULTS: No significant differences were detected in mortality at 24 hours (12.7% in 1:1:1 group vs 17.0% in 1:1:2 group; difference, -4.2% [95% CI, -9.6% to 1.1%]; P = .12) or at 30 days (22.4% vs 26.1%, respectively; difference, -3.7% [95% CI, -10.2% to 2.7%]; P = .26). Exsanguination, which was the predominant cause of death within the first 24 hours, was significantly decreased in the 1:1:1 group (9.2% vs 14.6% in 1:1:2 group; difference, -5.4% [95% CI, -10.4% to -0.5%]; P = .03). More patients in the 1:1:1 group achieved hemostasis than in the 1:1:2 group (86% vs 78%, respectively; P = .006). Despite the 1:1:1 group receiving more plasma (median of 7 U vs 5 U, P < .001) and platelets (12 U vs 6 U, P < .001) and similar amounts of red blood cells (9 U) over the first 24 hours, no differences between the 2 groups were found for the 23 prespecified complications, including acute respiratory distress syndrome, multiple organ failure, venous thromboembolism, sepsis, and transfusion-related complications. CONCLUSIONS AND RELEVANCE: Among patients with severe trauma and major bleeding, early administration of plasma, platelets, and red blood cells in a 1:1:1 ratio compared with a 1:1:2 ratio did not result in significant differences in mortality at 24 hours or at 30 days. However, more patients in the 1:1:1 group achieved hemostasis and fewer experienced death due to exsanguination by 24 hours. Even though there was an increased use of plasma and platelets transfused in the 1:1:1 group, no other safety differences were identified between the 2 groups. TRIAL REGISTRATION: clinicaltrials.gov Identifier: NCT01545232.

Association of the 2011 ACGME resident duty hour reform with general surgery patient outcomes and with resident examination performance.

IMPORTANCE: In 2011, the Accreditation Council for Graduate Medical Education (ACGME) restricted resident duty hour requirements beyond those established in 2003, leading to concerns about the effects on patient care and resident training. OBJECTIVE: To determine if the 2011 ACGME duty hour reform was associated with a change in general surgery patient outcomes or in resident examination performance. DESIGN, SETTING, AND PARTICIPANTS: Quasi-experimental study of general surgery patient outcomes 2 years before (academic years 2009-2010) and after (academic years 2012-2013) the 2011 duty hour reform. Teaching and nonteaching hospitals were compared using a difference-in-differences approach adjusted for procedural mix, patient comorbidities, and time trends. Teaching hospitals were defined based on the proportion of cases at which residents were present intraoperatively. Patients were those undergoing surgery at hospitals participating in the American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP). General surgery resident performance on the annual in-training, written board, and oral board examinations was assessed for this same period. EXPOSURES: National implementation of revised resident duty hour requirements on July 1, 2011, in all ACGME accredited residency programs. MAIN OUTCOMES AND MEASURES: Primary outcome was a composite of death or serious morbidity; secondary outcomes were other postoperative complications and resident examination performance. RESULTS: In the main analysis, 204,641 patients were identified from 23 teaching (n = 102,525) and 31 nonteaching (n = 102,116) hospitals. The unadjusted rate of death or serious morbidity improved during the study period in both teaching (11.6% [95% CI, 11.3%-12.0%] to 9.4% [95% CI, 9.1%-9.8%], P < .001) and nonteaching hospitals (8.7% [95% CI, 8.3%-9.0%] to 7.1% [95% CI, 6.8%-7.5%], P < .001). In adjusted analyses, the 2011 ACGME duty hour reform was not associated with a significant change in death or serious morbidity in either postreform year 1 (OR, 1.12; 95% CI, 0.98-1.28) or postreform year 2 (OR, 1.00; 95% CI, 0.86-1.17) or when both postreform years were combined (OR, 1.06; 95% CI, 0.93-1.20). There was no association between duty hour reform and any other postoperative adverse outcome. Mean (SD) in-training examination scores did not significantly change from 2010 to 2013 for first-year residents (499.7 [ 85.2] to 500.5 [84.2], P = .99), for residents from other postgraduate years, or for first-time examinees taking the written or oral board examinations during this period. CONCLUSIONS AND RELEVANCE: Implementation of the 2011 ACGME duty hour reform was not associated with a change in general surgery patient outcomes or differences in resident examination performance. The implications of these findings should be considered when evaluating the merit of the 2011 ACGME duty hour reform and revising related policies in the future.

Identifying Population-Level and Within-Hospital Disparities in Surgical Care.

INTRODUCTION: The lack of consensus on equity measurement and its incorporation into quality-assessment programs at the hospital and system levels may be a barrier to addressing disparities in surgical care. This study aimed to identify population-level and within-hospital differences in the quality of surgical care provision. METHODS: The analysis included 657 National Surgical Quality Improvement Program participating hospitals with over 4 million patients (2014-2018). Multi-level random slope, random intercept modeling was used to examine for population-level and in-hospital disparities. Disparities in surgical care by Area Deprivation Index (ADI), race, and ethnicity were analyzed for five measures: all-case inpatient mortality, all-case urgent readmission, all-case postoperative surgical site infection, colectomy mortality, and spine surgery complications. RESULTS: Population-level disparities were identified across all measures by ADI, two measures for Black race (all-case readmissions and spine surgery complications), and none for Hispanic ethnicity. Disparities remained significant in the adjusted models. Prior to risk-adjustment, in all measures examined, within-hospital disparities were detected in: 25.8-99.8% of hospitals for ADI, 0-6.1% of hospitals for Black race, and 0-0.8% of hospitals for Hispanic ethnicity. Following risk-adjustment, in all measures examined, fewer than 1.1% of hospitals demonstrated disparities by ADI, race, or ethnicity. CONCLUSIONS: Following risk adjustment, very few hospitals demonstrated significant disparities in care. Disparities were more frequently detected by ADI than by race and ethnicity. The lack of substantial in-hospital disparities may be due to the use of postoperative metrics, small sample sizes, the risk adjustment methodology, and healthcare segregation. Further work should examine surgical access and healthcare segregation.

Disparity-Sensitive Measures in Surgical Care: A Delphi Panel Consensus.

BACKGROUND: In the US, disparities in surgical care impede the delivery of uniformly high-quality care to all patients. There is a lack of disparity-sensitive measures related to surgical care. The American College of Surgeons Metrics for Equitable Access and Care in Surgery group, through research and expert consensus, aimed to identify disparity-sensitive measures in surgical care. STUDY DESIGN: An environmental scan, systematic literature review, and subspecialty society surveys were conducted to identify potential disparity-sensitive surgical measures. A modified Delphi process was conducted where panelists rated measures on both importance and validity. In addition, a novel literature-based disparity-sensitive scoring process was used. RESULTS: We identified 841 potential disparity-sensitive surgical measures. From these, our Delphi and literature-based approaches yielded a consensus list of 125 candidate disparity-sensitive measures. These measures were rated as both valid and important and were supported by the existing literature. CONCLUSION: There are profound disparities in surgical care within the US healthcare system. A multidisciplinary Delphi panel identified 125 potential disparity-sensitive surgical measures that could be used to track health disparities, evaluate the impact of focused interventions, and reduce healthcare inequity.

Development and Validation of a Novel Literature-Based Method to Identify Disparity-Sensitive Surgical Quality Metrics.

BACKGROUND: Disparity in surgical care impedes the delivery of uniformly high-quality care. Metrics that quantify disparity in care can help identify areas for needed intervention. A literature-based Disparity-Sensitive Score (DSS) system for surgical care was adapted by the Metrics for Equitable Access and Care in Surgery (MEASUR) group. The alignment between the MEASUR DSS and Delphi ratings of an expert advisory panel (EAP) regarding the disparity sensitivity of surgical quality metrics was assessed. STUDY DESIGN: Using DSS criteria MEASUR co-investigators scored 534 surgical metrics which were subsequently rated by the EAP. All scores were converted to a 9-point scale. Agreement between the new measurement technique (ie DSS) and an established subjective technique (ie importance and validity ratings) were assessed using the Bland-Altman method, adjusting for the linear relationship between the paired difference and the paired average. The limit of agreement (LOA) was set at 1.96 SD (95%). RESULTS: The percentage of DSS scores inside the LOA was 96.8% (LOA, 0.02 points) for the importance rating and 94.6% (LOA, 1.5 points) for the validity rating. In comparison, 94.4% of the 2 subjective EAP ratings were inside the LOA (0.7 points). CONCLUSIONS: Applying the MEASUR DSS criteria using available literature allowed for identification of disparity-sensitive surgical metrics. The results suggest that this literature-based method of selecting quality metrics may be comparable to more complex consensus-based Delphi methods. In fields with robust literature, literature-based composite scores may be used to select quality metrics rather than assembling consensus panels.