Controls, comparator arms, and designs for critical care comparative effectiveness research: It's complicated.

BACKGROUND: Comparative effectiveness research is meant to determine which commonly employed medical interventions are most beneficial, least harmful, and/or most costly in a real-world setting. While the objectives for comparative effectiveness research are clear, the field has failed to develop either a uniform definition of comparative effectiveness research or an appropriate set of recommendations to provide standards for the design of critical care comparative effectiveness research trials, spurring controversy in recent years. The insertion of non-representative control and/or comparator arm subjects into critical care comparative effectiveness research trials can threaten trial subjects' safety. Nonetheless, the broader scientific community does not always appreciate the importance of defining and maintaining critical care practices during a trial, especially when vulnerable, critically ill populations are studied. Consequently, critical care comparative effectiveness research trials sometimes lack properly constructed control or active comparator arms altogether and/or suffer from the inclusion of "unusual critical care" that may adversely affect groups enrolled in one or more arms. This oversight has led to critical care comparative effectiveness research trial designs that impair informed consent, confound interpretation of trial results, and increase the risk of harm for trial participants. METHODS/EXAMPLES: We propose a novel approach to performing critical care comparative effectiveness research trials that mandates the documentation of critical care practices prior to trial initiation. We also classify the most common types of critical care comparative effectiveness research trials, as well as the most frequent errors in trial design. We present examples of these design flaws drawn from past and recently published trials as well as examples of trials that avoided those errors. Finally, we summarize strategies employed successfully in well-designed trials, in hopes of suggesting a comprehensive standard for the field. CONCLUSION: Flawed critical care comparative effectiveness research trial designs can lead to unsound trial conclusions, compromise informed consent, and increase risks to research subjects, undermining the major goal of comparative effectiveness research: to inform current practice. Well-constructed control and comparator arms comprise indispensable elements of critical care comparative effectiveness research trials, key to improving the trials' safety and to generating trial results likely to improve patient outcomes in clinical practice.

Modeling current practices in critical care comparative effectiveness research.

Objective: To determine whether contemporaneous practices are adequately represented in recent critical care comparative effectiveness research studies. Design: All critical care comparative effectiveness research trials published in the New England Journal of Medicine from April 2019 to March 2020 were identified. To examine studies published in other high impact medical journals during the same period, such trials were subsequently also identified in the Journal of the American Medical Association and The Lancet. All cited sources were reviewed, and the medical literature was searched to find studies describing contemporary practices. Then, the designated control group or the comparable therapies studied were examined to determine if they represented contemporaneous critical care practices as described in the medical literature. Results: Twenty-five of 332 randomised clinical trials published in these three journals during this 1-year period described critical care comparative effectiveness research that met our inclusion criteria. Seventeen characterised current practices before enrolment (using surveys, observational studies and guidelines) and then incorporated current practices into one or more study arm. In the other eight, usual care arms appeared insufficient. Four of these trials randomly assigned patients to one of two fixed approaches at either end of a range of usually titrated care. However, due to randomisation, different subgroups within each arm received care that was inappropriate for their specific clinical conditions. In the other four of these trials, common practices influencing treatment choice were not reflected in the trial design, despite a prior effort to characterise usual care. Conclusion: One-third of critical care comparative effectiveness research trials published in widely read medical journals during a recent year did not include a designated control arm or comparable therapies representative of contemporary practices. Failure to incorporate contemporary practices into critical care comparative effectiveness trials appears to be a widespread design weakness.

Mechanistic insights into cell-free hemoglobin-induced injury during septic shock.

Cell-free hemoglobin (CFH) levels are elevated in septic shock and are higher in nonsurvivors. Whether CFH is only a marker of sepsis severity or is involved in pathogenesis is unknown. This study aimed to investigate whether CFH worsens sepsis-associated injuries and to determine potential mechanisms of harm. Fifty-one, 10-12 kg purpose-bred beagles were randomized to receive Staphylococcus aureus intrapulmonary challenges or saline followed by CFH infusions (oxyhemoglobin >80%) or placebo. Animals received antibiotics and intensive care support for 96 h. CFH significantly increased mean pulmonary arterial pressures and right ventricular afterload in both septic and nonseptic animals, effects that were significantly greater in nonsurvivors. These findings are consistent with CFH-associated nitric oxide (NO) scavenging and were associated with significantly depressed cardiac function, and worsened shock, lactate levels, metabolic acidosis, and multiorgan failure. In septic animals only, CFH administration significantly increased mean alveolar-arterial oxygenation gradients, also to a significantly greater degree in nonsurvivors. CFH-associated iron levels were significantly suppressed in infected animals, suggesting that bacterial iron uptake worsened pneumonia. Notably, cytokine levels were similar in survivors and nonsurvivors and were not predictive of outcome. In the absence and presence of infection, CFH infusions resulted in pulmonary hypertension, cardiogenic shock, and multiorgan failure, likely through NO scavenging. In the presence of infection alone, CFH infusions worsened oxygen exchange and lung injury, presumably by supplying iron that promoted bacterial growth. CFH elevation, a known consequence of clinical septic shock, adversely impacts sepsis outcomes through more than one mechanism, and is a biologically plausible, nonantibiotic, noncytokine target for therapeutic intervention.NEW & NOTEWORTHY Cell-free hemoglobin (CFH) elevations are a known consequence of clinical sepsis. Using a two-by-two factorial design and extensive physiological and biochemical evidence, we found a direct mechanism of injury related to nitric oxide scavenging leading to pulmonary hypertension increasing right heart afterload, depressed cardiac function, worsening circulatory failure, and death, as well as an indirect mechanism related to iron toxicity. These discoveries alter conventional thinking about septic shock pathogenesis and provide novel therapeutic approaches.

Comparative effectiveness research in critically ill patients: risks associated with mischaracterising usual care.

Comparative effectiveness research can help guide the use of common, routine medical practices. However, to be safe and informative, such trials must include at least one treatment arm that accurately portrays current practices. While comparative effectiveness research is widely perceived as safe and to involve no or only minimal risks, these assumptions may not hold true if unrecognised deviations from usual care exist in one or more study arms. For critically ill subjects in particular, such practice deviations may increase the risk of death or injury and undermine safety monitoring. Furthermore, unrecognised unusual care seems likely to corrupt informed consent documents, with underappreciated risks shrouded under the reassuring "comparative effectiveness" research label. At present, oversight measures are inadequate to ensure that research subjects enrolled in comparative effectiveness trials are actually receiving usual and not unusual care. Oversight by governmental and non-governmental entities with appropriate expertise, empowered to ensure that current clinical practice has been properly represented, could help prevent occurrences in clinical trials of unusual care masquerading as usual care.

Driving blind: instituting SEP-1 without high quality outcomes data.

In 2015, the Centers for Medicare and Medicaid Services (CMS) instituted an all-or-none sepsis performance measure bundle (SEP-1) to promote high-quality, cost-effective care. Systematic reviews demonstrated only low-quality evidence supporting most of SEP-1's interventions. CMS has removed some but not all of these unproven components. The current SEP-1 version requires patients with suspected sepsis have a lactate level, blood cultures, broad-spectrum antibiotics and, if hypotensive, a fixed 30 mL/kg fluid infusion within 3 hours, and a repeat lactate if initially elevated within 6 hours. Experts have continued to raise concerns that SEP-1 remains overly prescriptive, lacks a sound scientific basis and presents risks (overuse of antibiotics and inappropriate fluids not titrated to need). To incentivize compliance with SEP-1, CMS now publicly publishes how often hospitals complete all interventions in individual patients. However, compliance measured across hospitals (5 studies, 48-2,851 hospitals) or patients (three studies, 110-851 patients) has been low (approximately 50%) which is not surprising given SEP-1's lack of scientific basis. The largest observational study (1,738 patients) reporting survival rates employing SEP-1 found they were not significantly improved with the measure (P=0.53) as did the next largest study (851 patients, adjusted survival odds ratio 1.36, 95% CI, 0.85 to 2.18). Two smaller observational studies (158 and 450 patients) reported SEP-1 improved unadjusted survival (P≤0.05) but were confounded either by baseline imbalances or by simultaneous introduction of a code sepsis protocol to improve compliance. Regardless, retrospective studies have well known biases related to non-randomized designs, uncontrolled data collection and failure to adjust for unrecognized influential variables. Such low-quality science should not be the basis for a national mandate compelling care for a rapidly lethal disease with a high mortality rate. Instead, SEP-1 should be based on high quality reproducible evidence from randomized controlled trials (RCT) demonstrating its benefit and thereby safety. Otherwise we risk not only doing harm but standardizing it.

In canine bacterial pneumonia circulating granulocyte counts determine outcome from donor cells.

BACKGROUND: In experimental canine septic shock, depressed circulating granulocyte counts were associated with a poor outcome and increasing counts with prophylactic granulocyte colony-stimulating factor (G-CSF) improved outcome. Therapeutic G-CSF, in contrast, did not improve circulating counts or outcome, and therefore investigation was undertaken to determine whether transfusing granulocytes therapeutically would improve outcome. STUDY DESIGN AND METHODS: Twenty-eight purpose-bred beagles underwent an intrabronchial Staphylococcus aureus challenge and 4 hours later were randomly assigned to granulocyte (40-100 × 109 cells) or plasma transfusion. RESULTS: Granulocyte transfusion significantly expanded the low circulating counts for hours compared to septic controls but was not associated with significant mortality benefit (1/14, 7% vs. 2/14, 14%, respectively; p = 0.29). Septic animals with higher granulocyte count at 4 hours (median [interquartile range] of 3.81 3.39-5.05] vs. 1.77 [1.25-2.50]) had significantly increased survival independent of whether they were transfused with granulocytes. In a subgroup analysis, animals with higher circulating granulocyte counts receiving donor granulocytes had worsened lung injury compared to septic controls. Conversely, donor granulocytes decreased lung injury in septic animals with lower counts. CONCLUSION: During bacterial pneumonia, circulating counts predict the outcome of transfusing granulocytes. With low but normal counts, transfusing granulocytes does not improve survival and injures the lung, whereas for animals with very low counts, but not absolute neutropenia, granulocyte transfusion improves lung function.

Misrepresenting "Usual Care" in Research: An Ethical and Scientific Error.

Comparative effectiveness studies, referred to here as "usual-care" trials, seek to compare current medical practices for the same medical condition. Such studies are presumed to be safe and involve only minimal risks. However, that presumption may be flawed if the trial design contains "unusual" care, resulting in potential risks to subjects and inaccurately informed consent. Three case studies described here did not rely on clinical evidence to ascertain contemporaneous practice. As a result, the investigators drew inaccurate conclusions, misinformed research participants, and subjects' safety was compromised. Before approving usual-care protocols, IRBs and scientific review committees should evaluate the quality and completeness of information documenting usual-care practices. Guidance from governmental oversight agencies regarding evidence-based documentation of current clinical practice could prevent similar occurrences in future usual-care trials. Accurate information is necessary to ensure that trials comply with government regulations that require minimizing research risks to subjects and accurate informed consent documents.

RBC Storage Lesion Studies in Humans and Experimental Models of Shock.

The finding of toxicity in a meta-analysis of observational clinical studies of transfused longer stored red blood cells (RBC) and ethical issues surrounding aging blood for human studies prompted us to develop an experimental model of RBC transfusion. Transfusing older RBCs during canine pneumonia increased mortality rates. Toxicity was associated with in vivo hemolysis with release of cell-free hemoglobin (CFH) and iron. CFH can scavenge nitric oxide, causing vasoconstriction and endothelial injury. Iron, an essential bacterial nutrient, can worsen infections. This toxicity was seen at commonly transfused blood volumes (2 units) and was altered by the severity of pneumonia. Washing longer-stored RBCs mitigated these detrimental effects, but washing fresh RBCs actually increased them. In contrast to septic shock, transfused longer stored RBCs proved beneficial in hemorrhagic shock by decreasing reperfusion injury. Intravenous iron was equivalent in toxicity to transfusion of longer stored RBCs and both should be avoided during infection. Storage of longer-stored RBCs at 2 °C instead of higher standard temperatures (4-6 °C) minimized the release of CFH and iron. Haptoglobin, a plasma protein that binds CFH and increases its clearance, minimizes the toxic effects of longer-stored RBCs during infection and is a biologically plausible novel approach to treat septic shock.

Haptoglobin therapy has differential effects depending on severity of canine septic shock and cell-free hemoglobin level.

BACKGROUND: During sepsis, higher plasma cell-free hemoglobin (CFH) levels portend worse outcomes. In sepsis models, plasma proteins that bind CFH improve survival. In our canine antibiotic-treated Staphylococcus aureus pneumonia model, with and without red blood cell (RBC) exchange transfusion, commercial human haptoglobin (Hp) concentrates bound and compartmentalized CFH intravascularly, increased CFH clearance, and lowered iron levels, improving shock, lung injury, and survival. We now investigate in our model how very high CFH levels and treatment time affect Hp's beneficial effects. MATERIALS AND METHODS: Two separate canine pneumonia sepsis Hp studies were undertaken: one with exchange transfusion of RBCs after prolonged storage to raise CFH to very high levels and another with rapidly lethal sepsis alone to shorten time to treat. All animals received continuous standard intensive care unit supportive care for 96 hours. RESULTS: Older RBCs markedly elevated plasma CFH levels and, when combined with Hp therapy, created supraphysiologic CFH-Hp complexes that did not increase CFH or iron clearance or improve lung injury and survival. In a rapidly lethal bacterial challenge model without RBC transfusion, Hp binding did not increase clearance of complexes or iron or show benefits seen previously in the less lethal model. DISCUSSION: High-level CFH-Hp complexes may impair clearance mechanisms and eliminate Hp's beneficial effect during sepsis. Rapidly lethal sepsis narrows the therapeutic window for CFH and iron clearance, also decreasing Hp's beneficial effects. In designing clinical trials, dosing and kinetics may be critical factors if Hp infusion is used to treat sepsis.

Antibiotic- and Fluid-Focused Bundles Potentially Improve Sepsis Management, but High-Quality Evidence Is Lacking for the Specificity Required in the Centers for Medicare and Medicaid Service's Sepsis Bundle (SEP-1).

OBJECTIVE: To address three controversial components in the Centers for Medicare and Medicaid Service's sepsis bundle for performance measure (SEP-1): antibiotics within 3 hours, a 30 mL/kg fluid infusion for all hypotensive patients, and repeat lactate measurements within 6 hours if initially elevated. We hypothesized that antibiotic- and fluid-focused bundles like SEP-1 would probably show benefit, but evidence supporting specific antibiotic timing, fluid dosing, or serial lactate requirements would not be concordant. Therefore, we performed a meta-analysis of studies of sepsis bundles like SEP-1. DATA SOURCES: PubMed, Embase, ClinicalTrials.gov through March 15, 2018. STUDY SELECTION: Studies comparing survival in septic adults receiving versus not receiving antibiotic- and fluid-focused bundles. DATA EXTRACTION: Two investigators (D.J.P., P.Q.E.). DATA SYNTHESIS: Seventeen observational studies (11,303 controls and 4,977 bundle subjects) met inclusion criteria. Bundles were associated with increased odds ratios of survival (odds ratio [95% CI]) in 15 studies with substantial heterogeneity (I = 61%; p < 0.01). Survival benefits were consistent in the five largest (1,697-12,486 patients per study) (1.20 [1.11-1.30]; I = 0%) and six medium-sized studies (167-1,029) (2.03 [1.52-2.71]; I = 8%) but not the six smallest (64-137) (1.25 [0.42-3.66]; I = 57%). Bundles were associated with similarly increased survival benefits whether requiring antibiotics within 1 hour (n = 7 studies) versus 3 hours (n = 8) versus no specified time (n = 2); or 30 mL/kg fluid (n = 7) versus another volume (≥ 2 L, n = 1; ≥ 20 mL/kg, n = 2; 1.5-2 L or 500 mL, n = 1 each; none specified, n = 4) (p = 0.19 for each comparison). In the only study employing serial lactate measurements, survival was not increased versus others. No study had a low risk of bias or assessed potential adverse bundle effects. CONCLUSIONS: Available studies support the notion that antibiotic- and fluid-focused sepsis bundles like SEP-1 improve survival but do not demonstrate the superiority of any specific antibiotic time or fluid volume or of serial lactate measurements. Until strong reproducible evidence demonstrates the safety and benefit of any fixed requirement for these interventions, the present findings support the revision of SEP-1 to allow flexibility in treatment according to physician judgment.

Inhaled nebulized nitrite and nitrate therapy in a canine model of hypoxia-induced pulmonary hypertension.

Dysfunction in the nitric oxide (NO) signaling pathway can lead to the development of pulmonary hypertension (PH) in mammals. Discovery of an alternative pathway to NO generation involving reduction from nitrate to nitrite and to NO has motivated the evaluation of nitrite as an alternative to inhaled NO for PH. In contrast, inhaled nitrate has not been evaluated to date, and potential benefits include a prolonged half-life and decreased risk of methemoglobinemia. In a canine model of acute hypoxia-induced PH we evaluated the effects of inhaled nitrate to reduce pulmonary arterial pressure (PAP). In a randomized controlled trial, inhaled nitrate was compared to inhaled nitrite and inhaled saline. Exhaled NO, PAP and systemic blood pressures were continuously monitored. Inhaled nitrite significantly decreased PAP and increased exhaled NO. In contrast, inhaled nitrate and inhaled saline did not decrease PAP or increase exhaled NO. Unexpectedly, we found that inhaled nitrite resulted in prolonged (>5 h) exhaled NO release, increase in nitrate venous/arterial levels and a late surge in venous nitrite levels. These findings do not support a therapeutic role for inhaled nitrate in PH but may have therapeutic implications for inhaled nitrite in various disease states.

Impact of different standard red blood cell storage temperatures on human and canine RBC hemolysis and chromium survival.

BACKGROUND: Storage temperature is a critical factor for maintaining red-blood cell (RBC) viability, especially during prolonged cold storage. The target range of 1 to 6°C was established decades ago and may no longer be optimal for current blood-banking practices. STUDY DESIGN AND METHODS: Human and canine RBCs were collected under standard conditions and stored in precision-controlled refrigerators at 2°C, 4°C, or 6°C. RESULTS: During 42-day storage, human and canine RBCs showed progressive increases in supernatant non-transferrin-bound iron, cell-free hemoglobin, base deficit, and lactate levels that were overall greater at 6°C and 4°C than at 2°C. Animals transfused with 7-day-old RBCs had similar plasma cell-free hemoglobin and non-transferrin-bound iron levels at 1 to 72 hours for all three temperature conditions by chromium-51 recovery analysis. However, animals transfused with 35-day-old RBCs stored at higher temperatures developed plasma elevations in non-transferrin-bound iron and cell-free hemoglobin at 24 and 72 hours. Despite apparent impaired 35-day storage at 4°C and 6°C compared to 2°C, posttransfusion chromium-51 recovery at 24 hours was superior at higher temperatures. This finding was confounded by a preparation artifact related to an interaction between temperature and storage duration that leads to removal of fragile cells with repeated washing of the radiolabeled RBC test sample and renders the test sample unrepresentative of the stored unit. CONCLUSIONS: RBCs stored at the lower bounds of the temperature range are less metabolically active and produce less anaerobic acidosis and hemolysis, leading to a more suitable transfusion product. The higher refrigeration temperatures are not optimal during extended RBC storage and may confound chromium viability studies.

Haptoglobin improves shock, lung injury, and survival in canine pneumonia.

During the last half-century, numerous antiinflammatory agents were tested in dozens of clinical trials and have proven ineffective for treating septic shock. The observation in multiple studies that cell-free hemoglobin (CFH) levels are elevated during clinical sepsis and that the degree of increase correlates with higher mortality suggests an alternative approach. Human haptoglobin binds CFH with high affinity and, therefore, can potentially reduce iron availability and oxidative activity. CFH levels are elevated over approximately 24-48 hours in our antibiotic-treated canine model of S. aureus pneumonia that simulates the cardiovascular abnormalities of human septic shock. In this 96-hour model, resuscitative treatments, mechanical ventilation, sedation, and continuous care are translatable to management in human intensive care units. We found, in this S. aureus pneumonia model inducing septic shock, that commercial human haptoglobin concentrate infusions over 48-hours bind canine CFH, increase CFH clearance, and lower circulating iron. Over the 96-hour study, this treatment was associated with an improved metabolic profile (pH, lactate), less lung injury, reversal of shock, and increased survival. Haptoglobin binding compartmentalized CFH to the intravascular space. This observation, in combination with increasing CFHs clearance, reduced available iron as a potential source of bacterial nutrition while decreasing the ability for CFH and iron to cause extravascular oxidative tissue injury. In contrast, haptoglobin therapy had no measurable antiinflammatory effect on elevations in proinflammatory C-reactive protein and cytokine levels. Haptoglobin therapy enhances normal host defense mechanisms in contrast to previously studied antiinflammatory sepsis therapies, making it a biologically plausible novel approach to treat septic shock.

Evidence Underpinning the Centers for Medicare & Medicaid Services' Severe Sepsis and Septic Shock Management Bundle (SEP-1): A Systematic Review.

This article has been corrected. To see what has changed, please read the Letter to the Editor and the authors' response. The original version (PDF) is appended to this article as a Supplement. Background: The Severe Sepsis and Septic Shock Early Management Bundle (SEP-1), the sepsis performance measure introduced in 2015 by the Centers for Medicare & Medicaid Services (CMS), requires the reporting of up to 5 hemodynamic interventions, as many as 141 tasks, and 3 hours to document for a single patient. Purpose: To evaluate whether moderate- or high-level evidence shows that use of the 2015 SEP-1 or its hemodynamic interventions improves survival in adults with sepsis. Data Sources: PubMed, Embase, Scopus, Web of Science, and ClinicalTrials.gov from inception to 28 November 2017 with no language restrictions. Study Selection: Randomized and observational studies of death among adults with sepsis who received versus those who did not receive either the entire SEP-1 bundle or 1 or more SEP-1 hemodynamic interventions, including serial lactate measurements; a fluid infusion of 30 mL/kg of body weight; and assessment of volume status and tissue perfusion with a focused examination, bedside cardiovascular ultrasonography, or fluid responsiveness testing. Data Extraction: Two investigators independently extracted study data and assessed each study's risk of bias; 4 authors rated level of evidence by consensus using CMS criteria published in 2013. High- or moderate-level evidence required studies to have no confounders and low risk of bias. Data Synthesis: Of 56 563 references, 20 studies (18 reports) met inclusion criteria. One single-center observational study reported lower in-hospital mortality after implementation of the SEP-1 bundle. Sixteen studies (2 randomized and 14 observational) reported increased survival with serial lactate measurements or 30-mL/kg fluid infusions. None of the 17 studies were free of confounders or at low risk of bias. In 3 randomized trials, fluid responsiveness testing did not alter survival. Limitations: Few trials, poor-quality and confounded studies, and no studies (with survival outcomes) of the focused examination or bedside cardiovascular ultrasonography. Use of the 2015 version of SEP-1 and 2013 version of CMS evidence criteria, both of which were updated in 2017. Conclusion: No high- or moderate-level evidence shows that SEP-1 or its hemodynamic interventions improve survival in adults with sepsis. Primary Funding Source: National Institutes of Health. (PROSPERO: CRD42016052716).

Proceedings of the Food and Drug Administration's public workshop on new red blood cell product regulatory science 2016.

The US Food and Drug Administration (FDA) held a workshop on red blood cell (RBC) product regulatory science on October 6 and 7, 2016, at the Natcher Conference Center on the National Institutes of Health (NIH) Campus in Bethesda, Maryland. The workshop was supported by the National Heart, Lung, and Blood Institute, NIH; the Department of Defense; the Office of the Assistant Secretary for Health, Department of Health and Human Services; and the Center for Biologics Evaluation and Research, FDA. The workshop reviewed the status and scientific basis of the current regulatory framework and the available scientific tools to expand it to evaluate innovative and future RBC transfusion products. A full record of the proceedings is available on the FDA website (http://www.fda.gov/BiologicsBloodVaccines/NewsEvents/WorkshopsMeetingsConferences/ucm507890.htm). The contents of the summary are the authors' opinions and do not represent agency policy.