Estimation of Population Average Treatment Effects in the FIRST Trial: Application of a Propensity Score-Based Stratification Approach.

OBJECTIVE/STUDY QUESTION: To estimate and compare sample average treatment effects (SATE) and population average treatment effects (PATE) of a resident duty hour policy change on patient and resident outcomes using data from the Flexibility in Duty Hour Requirements for Surgical Trainees Trial ("FIRST Trial"). DATA SOURCES/STUDY SETTING: Secondary data from the National Surgical Quality Improvement Program and the FIRST Trial (2014-2015). STUDY DESIGN: The FIRST Trial was a cluster-randomized pragmatic noninferiority trial designed to evaluate the effects of a resident work hour policy change to permit greater flexibility in scheduling on patient and resident outcomes. We estimated hierarchical logistic regression models to estimate the SATE of a policy change on outcomes within an intent-to-treat framework. Propensity score-based poststratification was used to estimate PATE. DATA COLLECTION/EXTRACTION METHODS: This study was a secondary analysis of previously collected data. PRINCIPAL FINDINGS: Although SATE estimates suggested noninferiority of outcomes under flexible duty hour policy versus standard policy, the noninferiority of a policy change was inconclusively noninferior based on PATE estimates due to imprecision. CONCLUSIONS: Propensity score-based poststratification can be valuable tools to address trial generalizability but may yield imprecise estimates of PATE when sparse strata exist.

National Cluster-Randomized Trial of Duty-Hour Flexibility in Surgical Training.

BACKGROUND: Concerns persist regarding the effect of current surgical resident duty-hour policies on patient outcomes, resident education, and resident well-being. METHODS: We conducted a national, cluster-randomized, pragmatic, noninferiority trial involving 117 general surgery residency programs in the United States (2014-2015 academic year). Programs were randomly assigned to current Accreditation Council for Graduate Medical Education (ACGME) duty-hour policies (standard-policy group) or more flexible policies that waived rules on maximum shift lengths and time off between shifts (flexible-policy group). Outcomes included the 30-day rate of postoperative death or serious complications (primary outcome), other postoperative complications, and resident perceptions and satisfaction regarding their well-being, education, and patient care. RESULTS: In an analysis of data from 138,691 patients, flexible, less-restrictive duty-hour policies were not associated with an increased rate of death or serious complications (9.1% in the flexible-policy group and 9.0% in the standard-policy group, P=0.92; unadjusted odds ratio for the flexible-policy group, 0.96; 92% confidence interval, 0.87 to 1.06; P=0.44; noninferiority criteria satisfied) or of any secondary postoperative outcomes studied. Among 4330 residents, those in programs assigned to flexible policies did not report significantly greater dissatisfaction with overall education quality (11.0% in the flexible-policy group and 10.7% in the standard-policy group, P=0.86) or well-being (14.9% and 12.0%, respectively; P=0.10). Residents under flexible policies were less likely than those under standard policies to perceive negative effects of duty-hour policies on multiple aspects of patient safety, continuity of care, professionalism, and resident education but were more likely to perceive negative effects on personal activities. There were no significant differences between study groups in resident-reported perception of the effect of fatigue on personal or patient safety. Residents in the flexible-policy group were less likely than those in the standard-policy group to report leaving during an operation (7.0% vs. 13.2%, P<0.001) or handing off active patient issues (32.0% vs. 46.3%, P<0.001). CONCLUSIONS: As compared with standard duty-hour policies, flexible, less-restrictive duty-hour policies for surgical residents were associated with noninferior patient outcomes and no significant difference in residents' satisfaction with overall well-being and education quality. (FIRST ClinicalTrials.gov number, NCT02050789.).

Development of the Flexibility in Duty Hour Requirements for Surgical Trainees (FIRST) Trial Protocol: A National Cluster-Randomized Trial of Resident Duty Hour Policies.

IMPORTANCE: Debate continues regarding whether to further restrict resident duty hour policies, but little high-level evidence is available to guide policy changes. OBJECTIVE: To inform decision making regarding duty hour policies, the Flexibility in Duty Hour Requirements for Surgical Trainees (FIRST) Trial is being conducted to evaluate whether changing resident duty hour policies to permit greater flexibility in work hours affects patient postoperative outcomes, resident education, and resident well-being. DESIGN, SETTING, AND PARTICIPANTS: Pragmatic noninferiority cluster-randomized trial of general surgery residency programs with 2 study arms. Participating in the study are Accreditation Council for Graduate Medical Education (ACGME)-approved US general surgery residency programs (n = 118), their affiliated hospitals (n = 154), surgical residents and program directors, and general surgery patients from July 1, 2014, to June 30, 2015, with additional patient safety outcomes collected through June 30, 2016. The data collection platform for patient outcomes is the American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP), thus only hospitals participating in the ACS NSQIP were included. INTERVENTIONS: In the usual care arm, programs adhered to current ACGME resident duty hour standards. In the intervention arm, programs were allowed to deviate from current standards regarding maximum shift lengths and minimum time off between shifts through an ACGME waiver. MAIN OUTCOMES AND MEASURES: Death or serious morbidity within 30 days of surgery measured through ACS NSQIP, as well as resident satisfaction and well-being measured through a survey delivered at the time of the 2015 American Board of Surgery in Training Examination (ABSITE). RESULTS: A total of 118 general surgery residency programs and 154 hospitals were enrolled in the FIRST Trial and randomized. Fifty-nine programs (73 hospitals) were randomized to the usual care arm and 59 programs (81 hospitals) were randomized to the intervention arm. Intent-to-treat analysis will be used to estimate the effectiveness of assignment to the intervention arm on patient outcomes, resident education, and resident well-being compared with the usual care arm. Several sensitivity analyses will be performed to determine whether there were differential effects when examining only inpatients, high-risk patients, and emergent/urgent cases. CONCLUSIONS AND RELEVANCE: To our knowledge, the FIRST Trial is the first national randomized clinical trial of duty hour policies. Results of this study may be informative to policymakers and other stakeholders engaged in restructuring graduate medical training to enhance the quality of patient care and resident education. TRIAL REGISTRATION: clinicaltrials.org Identifier: NCT02050789.