Casting New Light on Statistical Power: An Illuminating Analogy and Strategies to Avoid Underpowered Trials.

Current standards for methodological rigor and trial reporting underscore the critical issue of statistical power. Still, the chance of detecting most effects reported in randomized controlled trials in medicine and other disciplines is currently lower than winning a toss of a fair coin. Here we propose that investigators who retain a practical understanding of how statistical power works can proactively avoid the potentially devastating consequences of underpowered trials. We first offer a vivid, carefully constructed analogy that illuminates the underlying relationships among 3 of the 5 essential parameters-namely, statistical power, effect size, and sample size-while holding the remaining 2 parameters constant (type of statistical test and significance level). Second, we extend the analogy to a set of critical scenarios in which investigators commonly miss detecting intervention effects due to insufficient statistical power. Third, we highlight effective pragmatic strategies for the design and conduct of sufficiently powered trials, without increasing sample size.

Downstream funding success of early career researchers for resubmitted versus new applications: A matched cohort.

BACKGROUND: Early career researchers face a hypercompetitive funding environment. To help identify effective intervention strategies for early career researchers, we examined whether first-time NIH R01 applicants who resubmitted their original, unfunded R01 application were more successful at obtaining any R01 funding within 3 and 5 years than original, unfunded applicants who submitted new NIH applications, and we examined whether underrepresented minority (URM) applicants differentially benefited from resubmission. Our observational study is consistent with an NIH working group's recommendations to develop interventions to encourage resubmission. METHODS AND FINDINGS: First-time applicants with US medical school academic faculty appointments who submitted an unfunded R01 application between 2000-2014 yielded 4,789 discussed and 7,019 not discussed applications. We then created comparable groups of first-time R01 applicants (resubmitted original R01 application or submitted new NIH applications) using optimal full matching that included applicant and application characteristics. Primary and subgroup analyses used generalized mixed models with obtaining any NIH R01 funding within 3 and 5 years as the two outcomes. A gamma sensitivity analysis was performed. URM applicants represented 11% and 12% of discussed and not discussed applications, respectively. First-time R01 applicants resubmitting their original, unfunded R01 application were more successful obtaining R01 funding within 3 and 5 years than applicants submitting new applications-for both discussed and not discussed applications: discussed within 3 years (OR 4.17 [95 CI 3.53, 4.93]) and 5 years (3.33 [2.82-3.92]); and not discussed within 3 years (2.81 [2.52, 3.13]) and 5 years (2.47 [2.22-2.74]). URM applicants additionally benefited within 5 years for not discussed applications. CONCLUSIONS: Encouraging early career researchers applying as faculty at a school of medicine to resubmit R01 applications is a promising potential modifiable factor and intervention strategy. First-time R01 applicants who resubmitted their original, unfunded R01 application had log-odds of obtaining downstream R01 funding within 3 and 5 years 2-4 times higher than applicants who did not resubmit their original application and submitted new NIH applications instead. Findings held for both discussed and not discussed applications.

Evaluation of an intervention targeted with predictive analytics to prevent readmissions in an integrated health system: observational study.

OBJECTIVES: To determine the associations between a care coordination intervention (the Transitions Program) targeted to patients after hospital discharge and 30 day readmission and mortality in a large, integrated healthcare system. DESIGN: Observational study. SETTING: 21 hospitals operated by Kaiser Permanente Northern California. PARTICIPANTS: 1 539 285 eligible index hospital admissions corresponding to 739 040 unique patients from June 2010 to December 2018. 411 507 patients were discharged post-implementation of the Transitions Program; 80 424 (19.5%) of these patients were at medium or high predicted risk and were assigned to receive the intervention after discharge. INTERVENTION: Patients admitted to hospital were automatically assigned to be followed by the Transitions Program in the 30 days post-discharge if their predicted risk of 30 day readmission or mortality was greater than 25% on the basis of electronic health record data. MAIN OUTCOME MEASURES: Non-elective hospital readmissions and all cause mortality in the 30 days after hospital discharge. RESULTS: Difference-in-differences estimates indicated that the intervention was associated with significantly reduced odds of 30 day non-elective readmission (adjusted odds ratio 0.91, 95% confidence interval 0.89 to 0.93; absolute risk reduction 95% confidence interval -2.5%, -3.1% to -2.0%) but not with the odds of 30 day post-discharge mortality (1.00, 0.95 to 1.04). Based on the regression discontinuity estimate, the association with readmission was of similar magnitude (absolute risk reduction -2.7%, -3.2% to -2.2%) among patients at medium risk near the risk threshold used for enrollment. However, the regression discontinuity estimate of the association with post-discharge mortality (-0.7% -1.4% to -0.0%) was significant and suggested benefit in this subgroup of patients. CONCLUSIONS: In an integrated health system, the implementation of a comprehensive readmissions prevention intervention was associated with a reduction in 30 day readmission rates. Moreover, there was no association with 30 day post-discharge mortality, except among medium risk patients, where some evidence for benefit was found. Altogether, the study provides evidence to suggest the effectiveness of readmission prevention interventions in community settings, but further research might be required to confirm the findings beyond this setting.

The Effect of Combining Business Training, Microfinance, and Support Group Participation on Economic Status and Intimate Partner Violence in an Unplanned Settlement of Nairobi, Kenya.

Intimate partner violence (IPV) has myriad negative health and economic consequences for women and families. We hypothesized that empowering women through a combination of formal business training, microfinance, and IPV support groups would decrease IPV and improve women's economic status. The study included adult female survivors of severe IPV. Women living in Korogocho received the intervention and women in Dandora served as a standard of care (SOC) group, but received the intervention at the end of the follow-up period. Women in the intervention groups (n = 82, SOC group, n = 81) received 8 weeks of business training, assistance creating a business plan, a small initial loan (about US$60), and weekly business and social support meetings. The two primary outcome measures included change in: (a) average daily profit margin, and (b) incidence of severe IPV. Exploratory analysis also looked at incidence of violence against children and women's self-efficacy. Average daily profit margin in the intervention group increased by 351 Kenyan Shillings (about US$3.5) daily (95% CI = [172, 485]). IPV directed against participating women decreased from a baseline of 2.1 to 0.26 incidents, a difference of 1.84 incidents (95% CI = [1.32, 2.36]). Violence against children in the household in the prior 3 months decreased from 1.1 to 0.55 incidents, a difference of 0.55 incidents (95% CI = [0.16, 1.03]). Finally, the intervention appears to have increased self-efficacy scores by 0.42 points (95% CIs 0.13, 0.71). In a low-resource urban environment, employing three complementary interventions resulted in higher daily profit margins and lower IPV in the intervention compared with the SOC group. These data support the notion that employing multiple interventions concomitantly might possess synergistic, beneficial effects, and hold promise to address profound poverty and interrupt the devastating cycle of IPV.

An evaluation of clinical order patterns machine-learned from clinician cohorts stratified by patient mortality outcomes.

OBJECTIVE: Evaluate the quality of clinical order practice patterns machine-learned from clinician cohorts stratified by patient mortality outcomes. MATERIALS AND METHODS: Inpatient electronic health records from 2010 to 2013 were extracted from a tertiary academic hospital. Clinicians (n = 1822) were stratified into low-mortality (21.8%, n = 397) and high-mortality (6.0%, n = 110) extremes using a two-sided P-value score quantifying deviation of observed vs. expected 30-day patient mortality rates. Three patient cohorts were assembled: patients seen by low-mortality clinicians, high-mortality clinicians, and an unfiltered crowd of all clinicians (n = 1046, 1046, and 5230 post-propensity score matching, respectively). Predicted order lists were automatically generated from recommender system algorithms trained on each patient cohort and evaluated against (i) real-world practice patterns reflected in patient cases with better-than-expected mortality outcomes and (ii) reference standards derived from clinical practice guidelines. RESULTS: Across six common admission diagnoses, order lists learned from the crowd demonstrated the greatest alignment with guideline references (AUROC range = 0.86-0.91), performing on par or better than those learned from low-mortality clinicians (0.79-0.84, P < 10-5) or manually-authored hospital order sets (0.65-0.77, P < 10-3). The same trend was observed in evaluating model predictions against better-than-expected patient cases, with the crowd model (AUROC mean = 0.91) outperforming the low-mortality model (0.87, P < 10-16) and order set benchmarks (0.78, P < 10-35). DISCUSSION: Whether machine-learning models are trained on all clinicians or a subset of experts illustrates a bias-variance tradeoff in data usage. Defining robust metrics to assess quality based on internal (e.g. practice patterns from better-than-expected patient cases) or external reference standards (e.g. clinical practice guidelines) is critical to assess decision support content. CONCLUSION: Learning relevant decision support content from all clinicians is as, if not more, robust than learning from a select subgroup of clinicians favored by patient outcomes.

Inpatient Clinical Order Patterns Machine-Learned From Teaching Versus Attending-Only Medical Services.

Clinical order patterns derived from data-mining electronic health records can be a valuable source of decision support content. However, the quality of crowdsourcing such patterns may be suspect depending on the population learned from. For example, it is unclear whether learning inpatient practice patterns from a university teaching service, characterized by physician-trainee teams with an emphasis on medical education, will be of variable quality versus an attending-only medical service that focuses strictly on clinical care. Machine learning clinical order patterns by association rule episode mining from teaching versus attending-only inpatient medical services illustrated some practice variability, but converged towards similar top results in either case. We further validated the automatically generated content by confirming alignment with external reference standards extracted from clinical practice guidelines.

Evidence That Classroom-Based Behavioral Interventions Reduce Pregnancy-Related School Dropout Among Nairobi Adolescents.

PURPOSE: To evaluate the effect of behavioral, empowerment-focused interventions on the incidence of pregnancy-related school dropout among girls in Nairobi's informal settlements. METHOD: Retrospective data on pregnancy-related school dropout from two cohorts were analyzed using a matched-pairs quasi-experimental design. The primary outcome was the change in the number of school dropouts due to pregnancy from 1 year before to 1 year after the interventions. RESULTS: Annual incidence of school dropout due to pregnancy decreased by 46% in the intervention schools (from 3.9% at baseline to 2.1% at follow-up), whereas the comparison schools remained essentially unchanged ( p < .029). Sensitivity analysis shows that the findings are robust to small levels of unobserved bias. CONCLUSIONS: Results suggest that these behavioral interventions significantly reduced the number of school dropouts due to pregnancy. As there are limited promising studies on behavioral interventions that decrease adolescent pregnancy in low-income settings, this intervention may be an important addition to this toolkit.

Health-related quality of life among veterans in addictions treatment: identifying behavioral targets for future intervention.

BACKGROUND: US veterans report lower health-related quality of life (HRQoL) relative to the general population. Identifying behavioral factors related to HRQoL that are malleable to change may inform interventions to improve well-being in this vulnerable group. PURPOSE: The current study sought to characterize HRQoL in a largely male sample of veterans in addictions treatment, both in relation to US norms and in association with five recommended health behavior practices: regularly exercising, managing stress, having good sleep hygiene, consuming fruits and vegetables, and being tobacco free. METHODS: We assessed HRQoL with 250 veterans in addictions treatment (96 % male, mean age 53, range 24-77) using scales from four validated measures. Data reduction methods identified two principal components reflecting physical and mental HRQoL. Model testing of HRQoL associations with health behaviors adjusted for relevant demographic and treatment-related covariates. RESULTS: Compared to US norms, the sample had lower HRQoL scores. Better psychological HRQoL was associated with higher subjective social standing, absence of pain or trauma, lower alcohol severity, and monotonically with the sum of health behaviors (all p < 0.05). Specifically, psychological HRQoL was associated with regular exercise, stress management, and sleep hygiene. Regular exercise also related to better physical HRQoL. The models explained >40 % of the variance in HRQoL. CONCLUSIONS: Exercise, sleep hygiene, and stress management are strongly associated with HRQoL among veterans in addictions treatment. Future research is needed to test the effect of interventions for improving well-being in this high-risk group.