2000 Years of Clinical Trials: The Evolution of Evidence-Based Practice.

ABSTRACT: Although the first documented clinical trial as described in the biblical book of Daniel dates to 606 BC, the prophet Daniel's nutrition study is contemporary in both approach and topic and could be considered the first comparative effectiveness research (CER) trial. This article summarizes the historical evolution of clinical trials and associated regulatory legislation. Ethical considerations foundational to nursing and evidence-based practice (EBP) in the 21st century are examined. Distinguishing features of CER, various study designs and checklists, and EBP are detailed. Biblical foundations for research and the Bible's relevance to modern research methods are discussed.

Use of the Poisson Distribution Is a Helpful Tool That Is Underused in Nursing Practice.

BACKGROUND: The Poisson distribution is used to find the probability of an event occurring over an interval of time, distance, area, or volume. PROBLEM: It is a helpful statistical tool, especially when evaluating rare events, and is underused in nursing practice. APPROACH: A single-group study design is used to demonstrate use of the Poisson distribution in determining whether a change in the number of discrete events is due to random variation or reflects a change in practice patterns and in determining the probability of seeing the number of observed events. OUTCOMES: Steps demonstrate how one can easily use the Poisson distribution to answer common questions. CONCLUSION: Use of the Poisson distribution can help nurses make better informed decisions about observed variations in care, especially when the data are not normally distributed, and can prevent undue concern when fluctuations in the number of events are associated with random fluctuations.

Risk of Distal Embolization From tPA (Tissue-Type Plasminogen Activator) Administration Prior to Endovascular Stroke Treatment.

BACKGROUND AND PURPOSE: In large artery occlusion stroke, both intravenous (IV) tPA (tissue-type plasminogen activator) and endovascular stroke treatment (EST) are standard-of-care. It is unknown how often tPA causes distal embolization, in which a procedurally accessible large artery occlusion is converted to a more distal and potentially inaccessible occlusion. METHODS: We analyzed data from a decentralized stroke telemedicine program in an integrated healthcare delivery system covering 21 hospitals, with 2 high-volume EST centers. We captured all cases sent for EST and examined the relationship between IV tPA administration and the rate of distal embolization, the rate of target recanalization (modified Treatment in Cerebral Infarction scale 2b/3), clinical improvement before EST, and short-term and long-term clinical outcomes. RESULTS: Distal embolization before EST was quite common (63/314 [20.1%]) and occurred more often after IV tPA before EST (57/229 [24.9%]) than among those not receiving IV tPA (6/85 [7.1%]; P<0.001). Distal embolization was associated with an inability to attempt EST: after distal embolization, 26/63 (41.3%) could not have attempted EST because of the new clot location, while in cases without distal embolization, only 8/249 (3.2%) were unable to have attempted EST (P<0.001). Among patients who received IV tPA, 13/242 (5.4%) had sufficient symptom improvement that a catheter angiogram was not performed; 6/342 (2.5%) had improvement to within 2 points of their baseline NIHSS. At catheter angiogram, 2/229 (0.9%) of patients who had received tPA had complete recanalization without distal embolization. Both IV tPA and EST recanalization were associated with improved long-term outcome. CONCLUSIONS: IV tPA administration before EST for large artery occlusion is associated with distal embolization, which in turn may reduce the chance that EST can be attempted and recanalization achieved. At the same time, some IV tPA-treated patients show symptomatic improvement and complete recanalization. Because IV tPA is associated with both distal embolization and improved long-term clinical outcome, there is a need for prospective clinical trials testing the net benefit or harm of IV tPA before EST.

Improved coding of postoperative deep vein thrombosis and pulmonary embolism in administrative data (AHRQ Patient Safety Indicator 12) after introduction of new ICD-9-CM diagnosis codes.

BACKGROUND: Symptomatic venous thromboembolism is a common postoperative complication. The Agency for Healthcare Research and Quality (AHRQ) has developed a Patient Safety Indicator 12 to assist hospitals, payers, and other stakeholders to identify patients who experienced this complication. OBJECTIVES: To determine whether newly created and recently redefined ICD-9-CM codes improved the criterion validity of Patient Safety Indicator 12, based on new samples of records dated after October 2009. RESEARCH DESIGN, SUBJECTS, MEASURES: Two sources of data were used: (1) UHC retrospective case-control study of risk factors for acute symptomatic venous thromboembolism occurring within 90 days after total knee arthroplasty in teaching hospitals; (2) chart abstraction data by volunteer hospitals participating in the Validation Pilot Project of the AHRQ. RESULTS: In the UHC sample, the positive predictive value (PPV) was 99% (125/126) and the negative predictive value was 99.4% (460/463). In the AHRQ sample, the overall PPV was 81% (126/156). CONCLUSIONS: The PPV based on both samples shows substantial improvement compared with the previously reported PPVs of 43%-48%, suggesting that changes in ICD-9-CM code architecture and better coding guidance can improve the usefulness of coded data.

Validity of the AHRQ patient safety indicator for postoperative physiologic and metabolic derangement based on a national sample of medical records.

OBJECTIVE: The Agency for Healthcare Research and Quality Patient Safety Indicator (PSI) 10, "Postoperative Physiologic and Metabolic Derangement" (PPMD), uses administrative data to detect postoperative acute kidney injury (AKI) requiring dialysis and diabetes-related complications. We sought to evaluate the indicator's criterion validity. RESEARCH DESIGN: We conducted a retrospective cross-sectional study of hospitalization records flagged positive and negative by PSI 10 from a diverse set of 35 hospitals between February 1, 2006 and June 30, 2009. Trained nurse abstractors reviewed medical records. We determined the indicator's sensitivity, specificity, and positive and negative predictive values. RESULTS: Of 94 records flagged by PSI 10 (87 for AKI, 7 for diabetic complications, 1 for both), 69 (73%) involved an accurately coded event; 60 (64%; 95% CI, 46%-79%) represented true PPMD from a clinical perspective. Two of 8 records flagged for diabetic complications were true events. Nineteen false positives involved preoperative renal failure. Three of 230 records flagged negative (enriched with questionably negative records) represented true PPMD. The indicator's sensitivity was 66% (20%-94%), specificity 99.9% (99.5%-100%), and negative predictive value 99.9% (99.4%-100%). Considering dialysis access procedures tantamount to dialysis and excluding records with lower urinary tract obstruction might increase the sensitivity and positive predictive value to 98% (87%-100%) and 72% (50%-87%), respectively. CONCLUSIONS: PSI 10 mostly concerns AKI and currently has moderate criterion validity, which might improve with increased use of "present on admission" coding, abandonment of the diabetes criteria, and adjustments to the indicator specifications regarding dialysis access and urinary tract obstruction.

Using the Agency for Healthcare Research and Quality patient safety indicators for targeting nursing quality improvement.

Quantifying the critical impact nurses have on the prevention and early recognition of potential complications and adverse events, such as those identified by the Agency for Healthcare Research and Quality (AHRQ) patient safety indicators (PSI), is becoming increasingly important. In this paper, we describe how the AHRQ PSI may be used to identify nursing-specific opportunities to improve care based on data from the national AHRQ PSI validation pilot project.

Incorporating harms into the weighting of the revised Agency for Healthcare Research and Quality Patient Safety for Selected Indicators Composite (Patient Safety Indicator 90).

OBJECTIVE: To reweight the Agency for Healthcare Research and Quality Patient Safety for Selected Indicators Composite (Patient Safety Indicator [PSI] 90) from weights based solely on the frequency of component PSIs to those that incorporate excess harm reflecting patients' preferences for outcome-related health states. DATA SOURCES: National administrative and claims data involving hospitalizations in nonfederal, nonrehabilitation, acute care hospitals. STUDY DESIGN: We estimated the average excess aggregate harm associated with the occurrence of each component PSI using a cohort sample for each indicator based on denominator-eligible records. We used propensity scores to account for potential confounding in the risk models for each PSI and weighted observations to estimate the "average treatment effect in the treated" for those with the PSI event. We fit separate regression models for each harm outcome. Final PSI weights reflected both the disutilities and the frequencies of the harms. DATA COLLECTION/EXTRACTION METHODS: We estimated PSI frequencies from the 2012 Healthcare Cost and Utilization Project State Inpatient Databases with present on admission data and excess harms using 2012-2013 Centers for Medicare & Medicaid Services Medicare Fee-for-Service data. PRINCIPAL FINDINGS: Including harms in the weighting scheme changed individual component weights from the original frequency-based weighting. In the reweighted composite, PSIs 11 ("Postoperative Respiratory Failure"), 13 ("Postoperative Sepsis"), and 12 ("Perioperative Pulmonary Embolism or Deep Vein Thrombosis") contributed the greatest harm, with weights of 29.7%, 21.1%, and 20.4%, respectively. Regarding reliability, the overall average hospital signal-to-noise ratio for the reweighted PSI 90 was 0.7015. Regarding discrimination, among hospitals with greater than median volume, 34% had significantly better PSI 90 performance, and 41% had significantly worse performance than benchmark rates (based on percentiles). CONCLUSIONS: Reformulation of PSI 90 with harm-based weights is feasible and results in satisfactory reliability and discrimination, with a more clinically meaningful distribution of component weights.

Designing an abstraction instrument: lessons from efforts to validate the AHRQ patient safety indicators.

BACKGROUND: The U.S. Agency for Healthcare Research and Quality (AHRQ) and other organizations have developed quality indicators based on hospital administrative data. Characteristics of effective abstraction instruments were identified for determining both the positive predictive value (PPV) of Patient Safety Indicators (PSIs) and the extent to which hospitals and clinicians could have prevented adverse events. METHODS: Through an iterative process involving nurse abstractors, physicians, and nurses with quality improvement experience, and health services researchers, 25 abstraction instruments were designed for 12 AHRQ provider-level morbidity PSIs. Data were analyzed from 13 of these instruments, and data are being collected using several more. FINDINGS: Common problems in designing the instruments included avoiding uninformative questions and premature termination of the abstraction process, anticipating misinterpretation of questions, allowing an appropriate range of response options; using clear terminology, optimizing the flow of the abstraction process, balancing the utility of data against abstractor burden, and recognizing the needs of end users, such as hospitals and quality improvement professionals and researchers, for the abstracted information. CONCLUSIONS: Designing medical record abstraction instruments for quality improvement research involves several potential pitfalls. Understanding how we addressed these challenges might help both investigators and users of outcome indicators to appreciate the strengths and limitations of outcome-based quality indicators and tools designed to validate or investigate such indicators within provider organizations.

Positive predictive value of the AHRQ accidental puncture or laceration patient safety indicator.

OBJECTIVE: Patient Safety Indicator (PSI) 15, or "Accidental Puncture or Laceration" (APL), of the US Agency for Healthcare Research and Quality was recently endorsed as a consensus standard for quality of care by the National Quality Forum. We sought to determine the positive predictive value (PPV) of this indicator. METHODS: We conducted a retrospective cross-sectional study of hospitalization records that met PSI 15 criteria. We sampled cases from 32 geographically diverse hospitals, including both teaching and nonteaching hospitals, between October 1, 2005 and March 31, 2007. Trained abstractors from each center reviewed randomly sampled medical records, using a standard instrument. We determined the PPV of the indicator and conducted descriptive analyses of the cases. RESULTS: Of the 249 cases that met PSI 15 criteria, 226 (91%; 95% CI: 88%-94%) represented true APL. Fifty-six of the true APL cases (24%) represented injuries that generally would be expected to heal without repair, yielding, from the standpoint of clinical relevance, a PPV of 68% (95% CI: 62%-74%). True positive cases that would typically warrant repair (n=170) were most likely to involve the gastrointestinal tract (30%), bladder (25%), dura (19%), or an important blood vessel (16%). In 97 of the true APL cases (43%), adhesions or other scar tissue were thought to have contributed to the complication. The 23 false-positive cases involved no apparent event other than normal operative conduct (n=7), a complication other than APL (bleeding, infection, dislodgement of a gastrostomy tube, or fracture) (7), an APL present on admission (5), or a disease-related lesion (4). CONCLUSIONS: Although PSI 15 is highly predictive of APL from a coding perspective, the indicator is less predictive of APL that could be considered clinically important. A significant proportion of cases represent relatively inconsequential injuries or injuries for which the risk may have been acceptable relative to the goals of the procedure.

How valid is the ICD-9-CM based AHRQ patient safety indicator for postoperative venous thromboembolism?

BACKGROUND: Hospital administrative data are being used to identify patients with postoperative venous thromboembolism (VTE), either pulmonary embolism (PE) or deep-vein thrombosis (DVT). However, few studies have evaluated the accuracy of these ICD-9-CM codes across multiple hospitals. METHODS AND MATERIALS: The Agency for Healthcare Research and Quality (AHRQ) Patient Safety Indicator (PSI)-12 was used to identify cases with postoperative VTE in 80 hospitals that volunteered for either an AHRQ or University HealthSystem Consortium (UHC) validation project. Trained abstractors using a standardized tool and guidelines retrospectively verified all coded VTE events. RESULTS: In the combined samples, the positive predictive value of the set of prespecified VTE codes for any acute VTE at any time during the hospitalization was 451 of 573 = 79% (95% CI: 75%-82%). However, the positive predictive value for acute lower extremity DVT or PE diagnosed after an operation was 209 of 452 = 44% (95% CI: 37%-51%) in the UHC sample and 58 of 121 = 48% (95% CI: 42-67%) in the AHRQ sample. Fourteen percent of all cases had an acute upper extremity DVT, 6% had superficial vein thrombosis and 21% had no acute VTE, however, 61% of the latter had a documented prior/chronic VTE. In the UHC cohort, the sensitivity for any acute VTE was 95.5% (95% CI: 86.4%-100%); the specificity was 99.5% (95% CI: 99.4%-99.7%). CONCLUSION: Current PSI 12 criteria do not accurately identify patients with acute postoperative lower extremity DVT or PE. Modification of the ICD-9-CM codes and implementation of "present on admission" flags should improve the predictive value for clinically important VTE events.

Angioedema Risk Associated With Central Vascular Access Device Declotting.

Little is known about the risk of angioedema with low-dose alteplase administration in the treatment of thrombotically occluded central vascular access devices (CVADs). To identify potential cases, the US Food and Drug Administration Adverse Event Reporting System database was searched. Between March 15, 2001, and August 15, 2018, there were 568 low-dose alteplase reports. Of these, 11 appear to be related to complications associated with a device or device occlusion that resulted in an angioedema-like reaction. This suggests that angioedema is a potential complication of alteplase when used for declotting CVADs-a complication that nurses should know how to recognize and treat.

Sex-Based Differences in Symptom Perception and Care-Seeking Behavior in Acute Stroke.

CONTEXT: Lack of early stroke recognition and delays in seeking emergency care by persons experiencing a stroke severely limit acute treatment options. Sparse and sometimes conflicting evidence suggests sex differences in care-seeking behaviors in stroke, stroke knowledge, perceptions of stroke symptoms, and the importance of physical location at the time of stroke and of having a witnessed stroke. OBJECTIVE: To explore specific sex-based differences in stroke presentation and associated care-seeking behaviors. DESIGN: Descriptive study based on a convenience sample of 60 patients with stroke admitted to an academic medical center in Northern California. MAIN OUTCOME MEASURES: Impact of the patient's sex on 1) time to presentation (early [≤ 4.5 hours] vs late [> 4.5 hours]); 2) perception of symptoms and clinical signs; 3) stroke knowledge and decision making; 4) physical location at the time of stroke; and 5) bystander presence and assistance with decision making. RESULTS: There was a discrepancy between how patients perceive symptoms and their medical findings on physical examination. Although most patients had at least one sign or symptom associated with nationally used stroke recognition acronyms, both sexes delayed care because they did not perceive their symptoms as urgent. Early-presenting men were more likely to have a higher stroke severity score at admission, receive alteplase, arrive by Emergency Medical Services, and have a witnessed stroke. Both early- and late-presenting women reported more nonfocal symptoms than did men. CONCLUSION: This study suggests that there are sex-based differences in symptom perception and care-seeking behavior in acute ischemic stroke.

How Accurate is the AHRQ Patient Safety Indicator for Hospital-Acquired Pressure Ulcer in a National Sample of Records?

In 2008, we conducted a retrospective cross-sectional study to determine the test characteristics of the Agency for Healthcare Research and Quality patient safety indicator (PSI) for hospital-acquired pressure ulcer (PU). We sampled 1,995 inpatient records that met PSI 3 criteria and 4,007 records assigned to 14 DRGs with the highest empirical rates of PSI 3, which did not meet PSI 3 criteria, from 32 U.S. academic hospitals. We estimated the positive predictive value (PPV), sensitivity, and specificity of PSI 3 using both the software version contemporary to the hospitalizations (v3.1) and an approximation of the current version (v4.4). Of records that met PSI 3 version 3.1 criteria, 572 (PPV 28.3%; 95% CI 23.6-32.9%) were true positive. PU that was present on admission (POA) accounted for 76% of the false-positive records. Estimated sensitivity was 48.2% (95% CI 41.0-55.3%) and specificity 71.4% (95% CI 68.3-74.5%). Reclassifying records based on reported POA information and PU stage to approximate version 4.4 of PSI 3 improved sensitivity (78.6%; 95% CI 62.7-94.5%) and specificity (98.0; 95% CI 97.1-98.9%). In conclusion, accounting for POA information and PU staging to approximate newer versions of the PSI software (v4.3) moderately improves validity.

Detecting postoperative hemorrhage or hematoma from administrative data: the performance of the AHRQ Patient Safety Indicator.

BACKGROUND: Patient Safety Indicator (PSI) 9, "postoperative hemorrhage or hematoma" (PHH), of the US Agency for Healthcare Research and Quality has been considered for public quality of care reporting. We sought to evaluate its performance in detecting true complications. METHODS: We conducted a retrospective, cross-sectional study of hospitalizations that met PSI 9 eligibility criteria. We sampled records flagged positive and negative by PSI 9 from a diverse set of 31 hospitals between February 2006, and June 2009. Trained abstractors reviewed medical records using standard instruments. We determined the sensitivity, specificity, and positive (PPV) and negative (NPV) predictive values of the indicator. RESULTS: Of 181 analyzable records flagged by PSI 9, 168 (93%; weighted PPV, 95% [95% confidence interval (CI), 90-98%]) involved an accurately coded event, but only 126 (70%; weighted PPV, 78% [95% CI, 58-90%]) represented true PHH. Thirty-two false positives involved only intraoperative hemorrhage. Among true positives, hypotension occurred in 28% and death attributed to the PHH in 4%. Thirty-two of 281 records flagged negative by PSI 9 (but enriched with questionably negative records) represented true PHH. The indicator's sensitivity was 42% (95% CI, 23-64%), specificity 99.9% (95% CI, 99.8-100%), and NPV 99.7% (95% CI, 99.0-99.9%). Modifying the indicator to include additional procedure codes improved both sensitivity (85% [95% CI, 67-94%]) and PPV (76% [95% CI, 60-88%]). CONCLUSION: PSI 9 holds promise in detecting serious, possibly preventable complications. The indicator might be improved by specification of the 998.11 hemorrhage code to exclude purely intraoperative events and addition of procedure codes to the indicator's numerator criteria.

Positive predictive value of the Agency for Healthcare Research and Quality Patient Safety Indicator for central line-related bloodstream infection ("selected infections due to medical care").

As part of the Agency for Healthcare Research and Quality Patient Safety Indicator (PSI) Validation Pilot Project, we evaluated the criterion validity of PSI 7. At the time of this study, PSI 7 was entitled "Selected Infections Due to Medical Care" and targeted catheter-related infections and inflammatory reactions. We conducted a retrospective cross-sectional study of 23 volunteer U.S. hospitals, where trained abstractors reviewed a sample of records that met PSI 7 criteria from October 1, 2005 to March 31, 2007. Of the 191 cases that met PSI 7 criteria, 104 (positive predictive value = 54%, 95% confidence interval: 40-69%) represented true infections. Of these cases, 77 (74%) were associated with central venous catheters, 15 (15%) were associated with peripheral intravenous (n=13) and or or arterial catheters (n=6), and 12 (11%) were associated with unknown catheters. Of the 87 (46%) false-positive cases, 41 (47%) did not have a qualifying infection identified by the abstractor, 38 (44%) had an infection present on admission, and 8 (9%) had an exclusionary diagnosis. PSI 7 has a low positive predictive value compared with other PSIs recently studied. Present on admission diagnoses and improved coding for infections related to central venous catheters (implemented October 2007) may improve validity.

Positive predictive value of the AHRQ Patient Safety Indicator "Postoperative Sepsis": implications for practice and policy.

BACKGROUND: Patient Safety Indicator (PSI) 13, or "Postoperative Sepsis," of the Agency for Healthcare Quality and Research (AHRQ), was recently adopted as part of a composite measure of patient safety by the Centers for Medicare and Medicaid Services (CMS). We sought to examine its positive predictive value (PPV) by determining how well it identifies true cases of postoperative sepsis. STUDY DESIGN: Two retrospective cross-sectional studies of hospitalization records that met PSI 13 criteria were conducted, one within the Veterans Administration (VA) Hospitals from fiscal years (FY) 2003 to 2007, and one within community hospitals between October 1, 2005 and March 31, 2007. Trained abstractors reviewed medical records from each database using standardized abstraction instruments. We determined the PPV of the indicator and performed descriptive analyses of cases. RESULTS: Of 112 cases flagged and reviewed within the VA system, 59 were true events of postoperative sepsis, yielding a PPV of 53% (95% CI 42% to 64%). Within the community hospital sector, of 164 flagged and reviewed cases, 67 were true cases of postoperative sepsis, yielding a PPV of 41% (95% CI 28% to 54%). False positives were due to infections that were present on admission, urgent or emergent cases, no clinical diagnosis of sepsis, or other coding limitations such as nonspecific shock in postoperative patients. CONCLUSIONS: PSI 13 has relatively poor predictive ability to identify true cases of postoperative sepsis in both the VA and nonfederal sectors. The lack of information on diagnosis timing, confusion about the definition of elective admission, and coding limitations were the major reasons for false positives. As it currently stands, the use of PSI 13 as a stand-alone measure for hospital reporting appears premature.