Evaluating site-of-care-related racial disparities in kidney graft failure using a novel federated learning framework.

OBJECTIVES: Racial disparities in kidney transplant access and posttransplant outcomes exist between non-Hispanic Black (NHB) and non-Hispanic White (NHW) patients in the United States, with the site of care being a key contributor. Using multi-site data to examine the effect of site of care on racial disparities, the key challenge is the dilemma in sharing patient-level data due to regulations for protecting patients' privacy. MATERIALS AND METHODS: We developed a federated learning framework, named dGEM-disparity (decentralized algorithm for Generalized linear mixed Effect Model for disparity quantification). Consisting of 2 modules, dGEM-disparity first provides accurately estimated common effects and calibrated hospital-specific effects by requiring only aggregated data from each center and then adopts a counterfactual modeling approach to assess whether the graft failure rates differ if NHB patients had been admitted at transplant centers in the same distribution as NHW patients were admitted. RESULTS: Utilizing United States Renal Data System data from 39 043 adult patients across 73 transplant centers over 10 years, we found that if NHB patients had followed the distribution of NHW patients in admissions, there would be 38 fewer deaths or graft failures per 10 000 NHB patients (95% CI, 35-40) within 1 year of receiving a kidney transplant on average. DISCUSSION: The proposed framework facilitates efficient collaborations in clinical research networks. Additionally, the framework, by using counterfactual modeling to calculate the event rate, allows us to investigate contributions to racial disparities that may occur at the level of site of care. CONCLUSIONS: Our framework is broadly applicable to other decentralized datasets and disparities research related to differential access to care. Ultimately, our proposed framework will advance equity in human health by identifying and addressing hospital-level racial disparities.

Learning from local to global: An efficient distributed algorithm for modeling time-to-event data.

OBJECTIVE: We developed and evaluated a privacy-preserving One-shot Distributed Algorithm to fit a multicenter Cox proportional hazards model (ODAC) without sharing patient-level information across sites. MATERIALS AND METHODS: Using patient-level data from a single site combined with only aggregated information from other sites, we constructed a surrogate likelihood function, approximating the Cox partial likelihood function obtained using patient-level data from all sites. By maximizing the surrogate likelihood function, each site obtained a local estimate of the model parameter, and the ODAC estimator was constructed as a weighted average of all the local estimates. We evaluated the performance of ODAC with (1) a simulation study and (2) a real-world use case study using 4 datasets from the Observational Health Data Sciences and Informatics network. RESULTS: On the one hand, our simulation study showed that ODAC provided estimates nearly the same as the estimator obtained by analyzing, in a single dataset, the combined patient-level data from all sites (ie, the pooled estimator). The relative bias was <0.1% across all scenarios. The accuracy of ODAC remained high across different sample sizes and event rates. On the other hand, the meta-analysis estimator, which was obtained by the inverse variance weighted average of the site-specific estimates, had substantial bias when the event rate is <5%, with the relative bias reaching 20% when the event rate is 1%. In the Observational Health Data Sciences and Informatics network application, the ODAC estimates have a relative bias <5% for 15 out of 16 log hazard ratios, whereas the meta-analysis estimates had substantially higher bias than ODAC. CONCLUSIONS: ODAC is a privacy-preserving and noniterative method for implementing time-to-event analyses across multiple sites. It provides estimates on par with the pooled estimator and substantially outperforms the meta-analysis estimator when the event is uncommon, making it extremely suitable for studying rare events and diseases in a distributed manner.

Testing small study effects in multivariate meta-analysis.

Small study effects occur when smaller studies show different, often larger, treatment effects than large ones, which may threaten the validity of systematic reviews and meta-analyses. The most well-known reasons for small study effects include publication bias, outcome reporting bias, and clinical heterogeneity. Methods to account for small study effects in univariate meta-analysis have been extensively studied. However, detecting small study effects in a multivariate meta-analysis setting remains an untouched research area. One of the complications is that different types of selection processes can be involved in the reporting of multivariate outcomes. For example, some studies may be completely unpublished while others may selectively report multiple outcomes. In this paper, we propose a score test as an overall test of small study effects in multivariate meta-analysis. Two detailed case studies are given to demonstrate the advantage of the proposed test over various naive applications of univariate tests in practice. Through simulation studies, the proposed test is found to retain nominal Type I error rates with considerable power in moderate sample size settings. Finally, we also evaluate the concordance between the proposed tests with the naive application of univariate tests by evaluating 44 systematic reviews with multiple outcomes from the Cochrane Database.

Engaging Patients and Other Non-Researchers in Health Research: Defining Research Engagement.

With the increase in patient and consumer activism through the late twentieth century and into this century, patient roles in research evolved into a new model of research engagement, with patients serving as active advisors and co-leading or leading clinical research. By requiring active engagement of patients and other stakeholders, several government research funders have advanced this model, particularly in Canada, the United States (US), United Kingdom (UK), and Australia. A consortium of individuals from these countries formed a Multi-Stakeholder Engagement (MuSE) consortium to examine critical issues in engaged research, establish consensus on definitions, and provide guidance for the field, beginning with an overview of how to involve stakeholders in health research (Concannon et al. J Gen Intern Med. 2019;34(3):458-463) and continuing here with an examination of definitions of research engagement. The political and advocacy roots of engaged research are reflected in definitions. Engagement is conceptualized with reference to research project goals, from informing specific clinical decisions to informing health-system level decisions. Political and cultural differences across countries are evident. Some of these government funders focus on empirical rather than ethical rationales. In countries with centralized health technology assessment, the link between societal values and engaged research is explicit. Ethical rationales for engagement are explicit in most of the published literature on research engagement. Harmonization of definitions is recommended so that research engagement elements, methods, and outcomes and impacts can be clearly examined and understood, and so that the field of research engagement can proceed from a clear conceptual foundation. Specific recommendations for terminology definitions are provided. Placing engaged research on a continuum from specific clinical decisions to more global public and social justice concerns clarifies the type of engaged research, supports appropriate comparisons, and improves the rigor of engaged research methods. The results help identify knowledge gaps in this growing field.

Comparative Clinical Effectiveness of Nonsurgical Treatment Methods in Patients With Lumbar Spinal Stenosis: A Randomized Clinical Trial.

Importance: Lumbar spinal stenosis (LSS) is the most common reason for spine surgery in older US adults. There is an evidence gap about nonsurgical LSS treatment options. Objective: To explore the comparative clinical effectiveness of 3 nonsurgical interventions for patients with LSS. Design, Setting, and Participants: Three-arm randomized clinical trial of 3 years' duration (November 2013 to June 2016). Analysis began in August 2016. All interventions were delivered during 6 weeks with follow-up at 2 months and 6 months at an outpatient research clinic. Patients older than 60 years with LSS were recruited from the general public. Eligibility required anatomical evidence of central canal and/or lateral recess stenosis (magnetic resonance imaging/computed tomography) and clinical symptoms associated with LSS (neurogenic claudication; less symptoms with flexion). Analysis was intention to treat. Interventions: Medical care, group exercise, and manual therapy/individualized exercise. Medical care consisted of medications and/or epidural injections provided by a physiatrist. Group exercise classes were supervised by fitness instructors in senior community centers. Manual therapy/individualized exercise consisted of spinal mobilization, stretches, and strength training provided by chiropractors and physical therapists. Main Outcomes and Measures: Primary outcomes were between-group differences at 2 months in self-reported symptoms and physical function measured by the Swiss Spinal Stenosis questionnaire (score range, 12-55) and a measure of walking capacity using the self-paced walking test (meters walked for 0 to 30 minutes). Results: A total of 259 participants (mean [SD] age, 72.4 [7.8] years; 137 women [52.9%]) were allocated to medical care (88 [34.0%]), group exercise (84 [32.4%]), or manual therapy/individualized exercise (87 [33.6%]). Adjusted between-group analyses at 2 months showed manual therapy/individualized exercise had greater improvement of symptoms and physical function compared with medical care (-2.0; 95% CI, -3.6 to -0.4) or group exercise (-2.4; 95% CI, -4.1 to -0.8). Manual therapy/individualized exercise had a greater proportion of responders (≥30% improvement) in symptoms and physical function (20%) and walking capacity (65.3%) at 2 months compared with medical care (7.6% and 48.7%, respectively) or group exercise (3.0% and 46.2%, respectively). At 6 months, there were no between-group differences in mean outcome scores or responder rates. Conclusions and Relevance: A combination of manual therapy/individualized exercise provides greater short-term improvement in symptoms and physical function and walking capacity than medical care or group exercises, although all 3 interventions were associated with improvements in long-term walking capacity. Trial Registration: ClinicalTrials.gov Identifier: NCT01943435.

Economic Evaluation of Quality Improvement Interventions Designed to Improve Glycemic Control in Diabetes: A Systematic Review and Weighted Regression Analysis.

OBJECTIVE: Quality improvement (QI) interventions can improve glycemic control, but little is known about their value. We systematically reviewed economic evaluations of QI interventions for glycemic control among adults with type 1 or type 2 diabetes. RESEARCH DESIGN AND METHODS: We used English-language studies from high-income countries that evaluated organizational changes and reported program and utilization-related costs, chosen from PubMed, EconLit, Centre for Reviews and Dissemination, New York Academy of Medicine's Grey Literature Report, and WorldCat (January 2004 to August 2016). We extracted data regarding intervention, study design, change in HbA1c, time horizon, perspective, incremental net cost (studies lasting ≤3 years), incremental cost-effectiveness ratio (ICER) (studies lasting ≥20 years), and study quality. Weighted least-squares regression analysis was used to estimate mean changes in HbA1c and incremental net cost. RESULTS: Of 3,646 records, 46 unique studies were eligible. Across 19 randomized controlled trials (RCTs), HbA1c declined by 0.26% (95% CI 0.17-0.35) or 3 mmol/mol (2 to 4) relative to usual care. In 8 RCTs lasting ≤3 years, incremental net costs were $116 (95% CI -$612 to $843) per patient annually. Long-term ICERs were $100,000-$115,000/quality-adjusted life year (QALY) in 3 RCTs, $50,000-$99,999/QALY in 1 RCT, $0-$49,999/QALY in 4 RCTs, and dominant in 1 RCT. Results were more favorable in non-RCTs. Our limitations include the fact that the studies had diverse designs and involved moderate risk of bias. CONCLUSIONS: Diverse multifaceted QI interventions that lower HbA1c appear to be a fair-to-good value relative to usual care, depending on society's willingness to pay for improvements in health.

A Payer-Guided Approach To Widespread Diffusion Of Behavioral Health Homes In Real-World Settings.

People with serious mental illness experience decreased life expectancy related to co-occurring medical conditions. A nonprofit behavioral health managed care organization implemented an innovative behavioral health home, in partnership with community mental health providers, to build a culture of wellness among all staff members, with a focus on prevention and holistic (that is, behavioral, social, and physical) health. The behavioral health home added one of two distinct care approaches, one patient driven and the other provider driven. The innovative approaches were implemented at eleven community mental health providers: Six delivered patient-driven care, and five delivered provider-driven care. We studied outcomes in the period October 2013-January 2016. Multiple diffusion strategies were utilized to encourage uptake. Our results revealed that both approaches significantly increased patient activation in care (more quickly in provider-supported care), engagement in primary and specialty care, and perceived mental health status. The success of this behavioral health home in improving important outcomes and the use of novel diffusion strategies led to its dissemination to forty-three additional providers across Pennsylvania.

AHRQ series on complex intervention systematic reviews-paper 5: advanced analytic methods.

BACKGROUND AND OBJECTIVE: Advanced analytic methods for synthesizing evidence about complex interventions continue to be developed. In this paper, we emphasize that the specific research question posed in the review should be used as a guide for choosing the appropriate analytic method. METHODS: We present advanced analytic approaches that address four common questions that guide reviews of complex interventions: (1) How effective is the intervention? (2) For whom does the intervention work and in what contexts? (3) What happens when the intervention is implemented? and (4) What decisions are possible given the results of the synthesis? CONCLUSION: The analytic approaches presented in this paper are particularly useful when each primary study differs in components, mechanisms of action, context, implementation, timing, and many other domains.

AHRQ series on complex intervention systematic reviews-paper 4: selecting analytic approaches.

BACKGROUND: Systematic reviews of complex interventions can vary widely in purpose, data availability and heterogeneity, and stakeholder expectations. RATIONALE: This article addresses the uncertainty that systematic reviewers face in selecting methods for reviews of complex interventions. Specifically, it lays out parameters for systematic reviewers to consider when selecting analytic approaches that best answer the questions at hand and suggests analytic techniques that may be appropriate in different circumstances. DISCUSSION: Systematic reviews of complex interventions comprising multiple questions may use multiple analytic approaches. Parameters to consider when choosing analytic methods for complex interventions include nature and timing of the decision (clinical practice guideline, policy, or other); purpose of the review; extent of existing evidence; logistic factors such as the timeline, process, and resources for deciding the scope of the review; and value of information to be obtained from choosing specific systematic review methods. Reviewers may elect to revise their analytic approach based on new or changing considerations during the course of the review but should guard against bias through transparency of reporting.

Caregiver Integration During Discharge Planning for Older Adults to Reduce Resource Use: A Metaanalysis.

OBJECTIVES: To determine the effect of integrating informal caregivers into discharge planning on postdischarge cost and resource use in older adults. DESIGN: A systematic review and metaanalysis of randomized controlled trials that examine the effect of discharge planning with caregiver integration begun before discharge on healthcare cost and resource use outcomes. MEDLINE, EMBASE, and the Cochrane Library databases were searched for all English-language articles published between 1990 and April 2016. SETTING: Hospital or skilled nursing facility. PARTICIPANTS: Older adults with informal caregivers discharged to a community setting. MEASUREMENTS: Readmission rates, length of and time to post-discharge rehospitalizations, costs of postdischarge care. RESULTS: Of 10,715 abstracts identified, 15 studies met the inclusion criteria. Eleven studies provided sufficient detail to calculate readmission rates for treatment and control participants. Discharge planning interventions with caregiver integration were associated with a 25% fewer readmissions at 90 days (relative risk (RR) = 0.75, 95% confidence interval (CI) = 0.62-0.91) and 24% fewer readmissions at 180 days (RR = 0.76, 95% CI = 0.64-0.90). The majority of studies reported statistically significant shorter time to readmission, shorter rehospitalization, and lower costs of postdischarge care among discharge planning interventions with caregiver integration. CONCLUSION: For older adults discharged to a community setting, the integration of caregivers into the discharge planning process reduces the risk of hospital readmission.

Association of Spinal Manipulative Therapy With Clinical Benefit and Harm for Acute Low Back Pain: Systematic Review and Meta-analysis.

IMPORTANCE: Acute low back pain is common and spinal manipulative therapy (SMT) is a treatment option. Randomized clinical trials (RCTs) and meta-analyses have reported different conclusions about the effectiveness of SMT. OBJECTIVE: To systematically review studies of the effectiveness and harms of SMT for acute (≤6 weeks) low back pain. DATA SOURCES: Search of MEDLINE, Cochrane Database of Systematic Reviews, EMBASE, and Current Nursing and Allied Health Literature from January 1, 2011, through February 6, 2017, as well as identified systematic reviews and RCTs, for RCTs of adults with low back pain treated in ambulatory settings with SMT compared with sham or alternative treatments, and that measured pain or function outcomes for up to 6 weeks. Observational studies were included to assess harms. DATA EXTRACTION AND SYNTHESIS: Data extraction was done in duplicate. Study quality was assessed using the Cochrane Back and Neck (CBN) Risk of Bias tool. This tool has 11 items in the following domains: randomization, concealment, baseline differences, blinding (patient), blinding (care provider [care provider is a specific quality metric used by the CBN Risk of Bias tool]), blinding (outcome), co-interventions, compliance, dropouts, timing, and intention to treat. Prior research has shown the CBN Risk of Bias tool identifies studies at an increased risk of bias using a threshold of 5 or 6 as a summary score. The evidence was assessed using the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) criteria. MAIN OUTCOMES AND MEASURES: Pain (measured by either the 100-mm visual analog scale, 11-point numeric rating scale, or other numeric pain scale), function (measured by the 24-point Roland Morris Disability Questionnaire or Oswestry Disability Index [range, 0-100]), or any harms measured within 6 weeks. FINDINGS: Of 26 eligible RCTs identified, 15 RCTs (1711 patients) provided moderate-quality evidence that SMT has a statistically significant association with improvements in pain (pooled mean improvement in the 100-mm visual analog pain scale, -9.95 [95% CI, -15.6 to -4.3]). Twelve RCTs (1381 patients) produced moderate-quality evidence that SMT has a statistically significant association with improvements in function (pooled mean effect size, -0.39 [95% CI, -0.71 to -0.07]). Heterogeneity was not explained by type of clinician performing SMT, type of manipulation, study quality, or whether SMT was given alone or as part of a package of therapies. No RCT reported any serious adverse event. Minor transient adverse events such as increased pain, muscle stiffness, and headache were reported 50% to 67% of the time in large case series of patients treated with SMT. CONCLUSIONS AND RELEVANCE: Among patients with acute low back pain, spinal manipulative therapy was associated with modest improvements in pain and function at up to 6 weeks, with transient minor musculoskeletal harms. However, heterogeneity in study results was large.

Challenges encountered in the conduct of Optimal Health: A patient-centered comparative effectiveness study of interventions for adults with serious mental illness.

BACKGROUND: The aim of patient-centered comparative effectiveness research is to conduct stakeholder-driven investigations that identify which interventions are most effective for which patients under specific circumstances. Conducting this research in real-world settings comes with unique experiences and challenges. We provide the study design, challenges confronted, and the solutions we devised for Optimal Health, a stakeholder-informed patient-centered comparative effectiveness study focused on the needs of seriously mentally ill individuals receiving case management services in community mental health centers across Pennsylvania. METHODS: Optimal Health, supported by the Patient-Centered Outcomes Research Institute, is a cluster-randomized trial of two evidence-based interventions for improving health and wellness across 11 provider sites. Participants were followed for 18-24 months, with repeated measurements of self-reported health status and activation in care and administrative measurements of primary and specialty health service utilization. Health-related quality of life, engagement in care, and service utilization are to be compared via random effects mixed models. Stakeholders were, and continue to be, engaged via focus groups, interviews, and stakeholder advisory board meetings. A learning collaborative model was used to support shared learning and implementation fidelity across provider sites. RESULTS: From 1 November 2013 through 15 July 2014, we recruited 1229 adults with serious mental illness, representing 85.1% of those eligible for study participation. Of these, 713 are in the Provider-Supported arm of the study and 516 in Patient Self-Directed Care. Across five data collection time points, we retained 86% and 83% of the participants in the Provider-Supported and Self-Directed arms, respectively. LESSONS LEARNED: Lessons learned relate to estimation of the size of our study population, the value of multiple data sources, and intervention training and implementation. The use of historical claims data can lead to an overestimation of eligible participants and, subsequently, a reduced study sample and an imbalance between intervention arms. Disruptions in continuity of care in real-world settings can pose challenges to on-site self-report data collection, although the inclusion of multiple data sources in study design can improve data completeness. Geographic dispersion of rural provider sites and staff turnover can lead to training and intervention fidelity challenges that can be overcome with the use of a "train-the-trainer" model, "wellness champions," and the use of a Learning Collaborative approach. Stakeholder engagement in mitigating these challenges proved to be critical to study progress. CONCLUSION: Conducting real-world patient-centered comparative effectiveness research in healthcare systems that care for seriously mentally ill persons is an important yet challenging undertaking, one which requires flexibility in identifying potential adaptations within all major study phases. Advice from a wide range of stakeholders is critical in development of successful strategies.

Association Between Palliative Care and Patient and Caregiver Outcomes: A Systematic Review and Meta-analysis.

Importance: The use of palliative care programs and the number of trials assessing their effectiveness have increased. Objective: To determine the association of palliative care with quality of life (QOL), symptom burden, survival, and other outcomes for people with life-limiting illness and for their caregivers. Data Sources: MEDLINE, EMBASE, CINAHL, and Cochrane CENTRAL to July 2016. Study Selection: Randomized clinical trials of palliative care interventions in adults with life-limiting illness. Data Extraction and Synthesis: Two reviewers independently extracted data. Narrative synthesis was conducted for all trials. Quality of life, symptom burden, and survival were analyzed using random-effects meta-analysis, with estimates of QOL translated to units of the Functional Assessment of Chronic Illness Therapy-palliative care scale (FACIT-Pal) instrument (range, 0-184 [worst-best]; minimal clinically important difference [MCID], 9 points); and symptom burden translated to the Edmonton Symptom Assessment Scale (ESAS) (range, 0-90 [best-worst]; MCID, 5.7 points). Main Outcomes and Measures: Quality of life, symptom burden, survival, mood, advance care planning, site of death, health care satisfaction, resource utilization, and health care expenditures. Results: Forty-three RCTs provided data on 12 731 patients (mean age, 67 years) and 2479 caregivers. Thirty-five trials used usual care as the control, and 14 took place in the ambulatory setting. In the meta-analysis, palliative care was associated with statistically and clinically significant improvements in patient QOL at the 1- to 3-month follow-up (standardized mean difference, 0.46; 95% CI, 0.08 to 0.83; FACIT-Pal mean difference, 11.36] and symptom burden at the 1- to 3-month follow-up (standardized mean difference, -0.66; 95% CI, -1.25 to -0.07; ESAS mean difference, -10.30). When analyses were limited to trials at low risk of bias (n = 5), the association between palliative care and QOL was attenuated but remained statistically significant (standardized mean difference, 0.20; 95% CI, 0.06 to 0.34; FACIT-Pal mean difference, 4.94), whereas the association with symptom burden was not statistically significant (standardized mean difference, -0.21; 95% CI, -0.42 to 0.00; ESAS mean difference, -3.28). There was no association between palliative care and survival (hazard ratio, 0.90; 95% CI, 0.69 to 1.17). Palliative care was associated consistently with improvements in advance care planning, patient and caregiver satisfaction, and lower health care utilization. Evidence of associations with other outcomes was mixed. Conclusions and Relevance: In this meta-analysis, palliative care interventions were associated with improvements in patient QOL and symptom burden. Findings for caregiver outcomes were inconsistent. However, many associations were no longer significant when limited to trials at low risk of bias, and there was no significant association between palliative care and survival.

Economic Evaluation of Quality Improvement Interventions for Bloodstream Infections Related to Central Catheters: A Systematic Review.

IMPORTANCE: Although quality improvement (QI) interventions can reduce central-line-associated bloodstream infections (CLABSI) and catheter-related bloodstream infections (CRBSI), their economic value is uncertain. OBJECTIVE: To systematically review economic evaluations of QI interventions designed to prevent CLABSI and/or CRBSI in acute care hospitals. EVIDENCE REVIEW: A search of Ovid MEDLINE, Econlit, Centre for Reviews & Dissemination, New York Academy of Medicine's Grey Literature Report, Worldcat, prior systematic reviews (January 2004 to July 2016), and IDWeek conference abstracts (2013-2016), was conducted from 2013 to 2016. We included English-language studies of any design that evaluated organizational or structural changes to prevent CLABSI or CRBSI, and reported program and infection-related costs. Dual reviewers assessed study design, effectiveness, costs, and study quality. For each eligible study, we performed a cost-consequences analysis from the hospital perspective, estimating the incidence rate ratio (IRR) and incremental net savings. Unadjusted weighted regression analyses tested predictors of these measures, weighted by catheter-days per study per year. FINDINGS: Of 505 articles, 15 unique studies were eligible, together representing data from 113 hospitals. Thirteen studies compared Agency for Healthcare Research and Quality-recommended practices with usual care, including 7 testing insertion checklists. Eleven studies were based on uncontrolled before-after designs, 1 on a randomized controlled trial, 1 on a time-series analysis, and 2 on modeled estimates. Overall, the weighted mean IRR was 0.43 (95% CI, 0.35-0.51) and incremental net savings were $1.85 million (95% CI, $1.30 million to $2.40 million) per hospital over 3 years (2015 US dollars). Each $100 000-increase in program cost was associated with $315 000 greater savings (95% CI, $166 000-$464 000; P < .001). Infections and net costs declined when hospitals already used checklists or had baseline infection rates of 1.7 to 3.7 per 1000 catheter-days. Study quality was not associated with effectiveness or costs. CONCLUSIONS AND RELEVANCE: Interventions related to central venous catheters were, on average, associated with 57% fewer bloodstream infections and substantial savings to hospitals. Larger initial investments may be associated with greater savings. Although checklists are now widely used and infections have started to decline, additional improvements and savings can occur at hospitals that have not yet attained very low infection rates.

Standards and guidelines for observational studies: quality is in the eye of the beholder.

OBJECTIVES: Patient care decisions demand high-quality research. To assist those decisions, numerous observational studies are being performed. Are the standards and guidelines to assess observational studies consistent and actionable? What policy considerations should be considered to ensure decision makers can determine if an observational study is of high-quality and valid to inform treatment decisions? STUDY DESIGN AND SETTING: Based on a literature review and input from six experts, we compared and contrasted nine standards/guidelines using 23 methodological elements involved in observational studies (e.g., study protocol, data analysis, and so forth). RESULTS: Fourteen elements (61%) were addressed by at least seven standards/guidelines; 12 of these elements disagreed in the approach. Nine elements (39%) were addressed by six or fewer standards/guidelines. Ten elements (43%) were not actionable in at least one standard/guideline that addressed the element. CONCLUSION: The lack of observational study standard/guideline agreement may contribute to variation in study conduct; disparities in what is considered credible research; and ultimately, what evidence is adopted. A common set of agreed on standards/guidelines for conducting observational studies will benefit funders, researchers, journal editors, and decision makers.

Evaluating Academic Scientists Collaborating in Team-Based Research: A Proposed Framework.

Criteria for evaluating faculty are traditionally based on a triad of scholarship, teaching, and service. Research scholarship is often measured by first or senior authorship on peer-reviewed scientific publications and being principal investigator on extramural grants. Yet scientific innovation increasingly requires collective rather than individual creativity, which traditional measures of achievement were not designed to capture and, thus, devalue. The authors propose a simple, flexible framework for evaluating team scientists that includes both quantitative and qualitative assessments. An approach for documenting contributions of team scientists in team-based scholarship, nontraditional education, and specialized service activities is also outlined. Although biostatisticians are used for illustration, the approach is generalizable to team scientists in other disciplines.The authors offer three key recommendations to members of institutional promotion committees, department chairs, and others evaluating team scientists. First, contributions to team-based scholarship and specialized contributions to education and service need to be assessed and given appropriate and substantial weight. Second, evaluations must be founded on well-articulated criteria for assessing the stature and accomplishments of team scientists. Finally, mechanisms for collecting evaluative data must be developed and implemented at the institutional level. Without these three essentials, contributions of team scientists will continue to be undervalued in the academic environment.

Systematic review and metaanalysis of middle meatal packing after endoscopic sinus surgery.

OBJECTIVE: The objective of this systematic review was to evaluate synechiae formation in patients who underwent packing in the middle meatus at the completion of endoscopic sinus surgery (ESS) for chronic rhinosinusitis (CRS) or recurrent acute rhinosinusitis. DATA SOURCES: PubMed and Ovid. REVIEW METHODS: Prospective randomized controlled studies of ESS with the placement of middle meatal packing were reviewed and included for analysis. Metaanalysis of the pooled data was performed. RESULTS: Eighteen prospective, randomized, controlled trials met inclusion criteria, resulting in a total of 925 subjects. Although there was a trend toward decreased risk of synechiae formation in the patients who underwent placement of middle meatal packing postoperatively (RR = 0.544), this did not achieve statistical significance on metaanalysis (p = 0.052). CONCLUSIONS: This systematic review and metaanalysis suggests that use of middle meatal packing does not significantly reduce the risk of synechiae formation after ESS. There was, however, significant heterogeneity of the data analyzed, and the difference between groups was just short of statistical significance in our metaanalysis. Additional prospective randomized studies on this topic will further elucidate the utility of middle meatal packing.

Grading the strength of a body of evidence when assessing health care interventions: an EPC update.

OBJECTIVES: To revise 2010 guidance on grading the strength of evidence (SOE) of the effectiveness of drugs, devices, and other preventive and therapeutic interventions in systematic reviews produced by the Evidence-based Practice Center (EPC) program, established by the US Agency for Healthcare Research and Quality (AHRQ). STUDY DESIGN AND SETTING: A cross-EPC working group reviewed authoritative systems for grading SOE [primarily the approach from the Grading of Recommendations Assessment, Development and Evaluation (GRADE) working group] and conducted extensive discussions with GRADE and other experts. RESULTS: Updated guidance continues to be conceptually similar to GRADE. Reviewers are to evaluate SOE separately for each major treatment comparison for each major outcome. We added reporting bias as a required domain and retained study limitations (risk of bias), consistency, directness, and precision (and three optional domains). Additional guidance covers scoring consistency, precision, and reporting bias, grading bodies of evidence with randomized controlled trials and observational studies, evaluating single study bodies of evidence, using studies with high risk of bias, and presenting findings with greater clarity and transparency. SOE is graded high, moderate, low, or insufficient, reflecting reviewers' confidence in the findings for a specific treatment comparison and outcome. CONCLUSION: No single approach for grading SOE suits all reviews, but a more consistent and transparent approach to reporting summary information will make reviews more useful to the broad range of audiences that AHRQ's work aims to reach. EPC working groups will consider ongoing challenges and modify guidance as needed, on issues such as combining trials and observational studies in bodies of evidence, weighting domains, and combining qualitative and quantitative syntheses.

Model for a patient-centered comparative effectiveness research center.

This special report describes the systematic approach the University of Pittsburgh and the University of Pittsburgh Medical Center (UPMC) undertook in creating an infrastructure for comparative effectiveness and patient-centered outcomes research resources. We specifically highlight the administrative structure, communication and training opportunities, stakeholder engagement resources, and support services offered.