RESUMO
OBJECTIVE: The Agency for Healthcare Research and Quality Patient Safety Indicator (PSI) 10, "Postoperative Physiologic and Metabolic Derangement" (PPMD), uses administrative data to detect postoperative acute kidney injury (AKI) requiring dialysis and diabetes-related complications. We sought to evaluate the indicator's criterion validity. RESEARCH DESIGN: We conducted a retrospective cross-sectional study of hospitalization records flagged positive and negative by PSI 10 from a diverse set of 35 hospitals between February 1, 2006 and June 30, 2009. Trained nurse abstractors reviewed medical records. We determined the indicator's sensitivity, specificity, and positive and negative predictive values. RESULTS: Of 94 records flagged by PSI 10 (87 for AKI, 7 for diabetic complications, 1 for both), 69 (73%) involved an accurately coded event; 60 (64%; 95% CI, 46%-79%) represented true PPMD from a clinical perspective. Two of 8 records flagged for diabetic complications were true events. Nineteen false positives involved preoperative renal failure. Three of 230 records flagged negative (enriched with questionably negative records) represented true PPMD. The indicator's sensitivity was 66% (20%-94%), specificity 99.9% (99.5%-100%), and negative predictive value 99.9% (99.4%-100%). Considering dialysis access procedures tantamount to dialysis and excluding records with lower urinary tract obstruction might increase the sensitivity and positive predictive value to 98% (87%-100%) and 72% (50%-87%), respectively. CONCLUSIONS: PSI 10 mostly concerns AKI and currently has moderate criterion validity, which might improve with increased use of "present on admission" coding, abandonment of the diabetes criteria, and adjustments to the indicator specifications regarding dialysis access and urinary tract obstruction.
Assuntos
Injúria Renal Aguda/epidemiologia , Complicações do Diabetes/epidemiologia , Segurança do Paciente/estatística & dados numéricos , Complicações Pós-Operatórias/epidemiologia , Indicadores de Qualidade em Assistência à Saúde/estatística & dados numéricos , United States Agency for Healthcare Research and Quality/estatística & dados numéricos , Injúria Renal Aguda/etiologia , Idoso , Idoso de 80 Anos ou mais , Estudos Transversais , Complicações do Diabetes/etiologia , Reações Falso-Negativas , Reações Falso-Positivas , Feminino , Humanos , Masculino , Pessoa de Meia-Idade , Complicações Pós-Operatórias/etiologia , Reprodutibilidade dos Testes , Estudos Retrospectivos , Estados UnidosRESUMO
BACKGROUND: The U.S. Agency for Healthcare Research and Quality (AHRQ) and other organizations have developed quality indicators based on hospital administrative data. Characteristics of effective abstraction instruments were identified for determining both the positive predictive value (PPV) of Patient Safety Indicators (PSIs) and the extent to which hospitals and clinicians could have prevented adverse events. METHODS: Through an iterative process involving nurse abstractors, physicians, and nurses with quality improvement experience, and health services researchers, 25 abstraction instruments were designed for 12 AHRQ provider-level morbidity PSIs. Data were analyzed from 13 of these instruments, and data are being collected using several more. FINDINGS: Common problems in designing the instruments included avoiding uninformative questions and premature termination of the abstraction process, anticipating misinterpretation of questions, allowing an appropriate range of response options; using clear terminology, optimizing the flow of the abstraction process, balancing the utility of data against abstractor burden, and recognizing the needs of end users, such as hospitals and quality improvement professionals and researchers, for the abstracted information. CONCLUSIONS: Designing medical record abstraction instruments for quality improvement research involves several potential pitfalls. Understanding how we addressed these challenges might help both investigators and users of outcome indicators to appreciate the strengths and limitations of outcome-based quality indicators and tools designed to validate or investigate such indicators within provider organizations.
Assuntos
Administração Hospitalar , Melhoria de Qualidade/organização & administração , Indicadores de Qualidade em Assistência à Saúde/organização & administração , Gestão da Segurança/organização & administração , Humanos , Reprodutibilidade dos Testes , Estados Unidos , United States Agency for Healthcare Research and QualityRESUMO
OBJECTIVE: Patient Safety Indicator (PSI) 15, or "Accidental Puncture or Laceration" (APL), of the US Agency for Healthcare Research and Quality was recently endorsed as a consensus standard for quality of care by the National Quality Forum. We sought to determine the positive predictive value (PPV) of this indicator. METHODS: We conducted a retrospective cross-sectional study of hospitalization records that met PSI 15 criteria. We sampled cases from 32 geographically diverse hospitals, including both teaching and nonteaching hospitals, between October 1, 2005 and March 31, 2007. Trained abstractors from each center reviewed randomly sampled medical records, using a standard instrument. We determined the PPV of the indicator and conducted descriptive analyses of the cases. RESULTS: Of the 249 cases that met PSI 15 criteria, 226 (91%; 95% CI: 88%-94%) represented true APL. Fifty-six of the true APL cases (24%) represented injuries that generally would be expected to heal without repair, yielding, from the standpoint of clinical relevance, a PPV of 68% (95% CI: 62%-74%). True positive cases that would typically warrant repair (n=170) were most likely to involve the gastrointestinal tract (30%), bladder (25%), dura (19%), or an important blood vessel (16%). In 97 of the true APL cases (43%), adhesions or other scar tissue were thought to have contributed to the complication. The 23 false-positive cases involved no apparent event other than normal operative conduct (n=7), a complication other than APL (bleeding, infection, dislodgement of a gastrostomy tube, or fracture) (7), an APL present on admission (5), or a disease-related lesion (4). CONCLUSIONS: Although PSI 15 is highly predictive of APL from a coding perspective, the indicator is less predictive of APL that could be considered clinically important. A significant proportion of cases represent relatively inconsequential injuries or injuries for which the risk may have been acceptable relative to the goals of the procedure.
Assuntos
Acidentes/estatística & dados numéricos , Hospitalização/estatística & dados numéricos , Lacerações/epidemiologia , Punções/estatística & dados numéricos , Indicadores de Qualidade em Assistência à Saúde , Gestão de Riscos/estatística & dados numéricos , Estudos Transversais , Feminino , Registros Hospitalares/estatística & dados numéricos , Humanos , Incidência , Masculino , Pessoa de Meia-Idade , Valor Preditivo dos Testes , Estudos Retrospectivos , Estados Unidos/epidemiologiaRESUMO
BACKGROUND: Hospital administrative data are being used to identify patients with postoperative venous thromboembolism (VTE), either pulmonary embolism (PE) or deep-vein thrombosis (DVT). However, few studies have evaluated the accuracy of these ICD-9-CM codes across multiple hospitals. METHODS AND MATERIALS: The Agency for Healthcare Research and Quality (AHRQ) Patient Safety Indicator (PSI)-12 was used to identify cases with postoperative VTE in 80 hospitals that volunteered for either an AHRQ or University HealthSystem Consortium (UHC) validation project. Trained abstractors using a standardized tool and guidelines retrospectively verified all coded VTE events. RESULTS: In the combined samples, the positive predictive value of the set of prespecified VTE codes for any acute VTE at any time during the hospitalization was 451 of 573 = 79% (95% CI: 75%-82%). However, the positive predictive value for acute lower extremity DVT or PE diagnosed after an operation was 209 of 452 = 44% (95% CI: 37%-51%) in the UHC sample and 58 of 121 = 48% (95% CI: 42-67%) in the AHRQ sample. Fourteen percent of all cases had an acute upper extremity DVT, 6% had superficial vein thrombosis and 21% had no acute VTE, however, 61% of the latter had a documented prior/chronic VTE. In the UHC cohort, the sensitivity for any acute VTE was 95.5% (95% CI: 86.4%-100%); the specificity was 99.5% (95% CI: 99.4%-99.7%). CONCLUSION: Current PSI 12 criteria do not accurately identify patients with acute postoperative lower extremity DVT or PE. Modification of the ICD-9-CM codes and implementation of "present on admission" flags should improve the predictive value for clinically important VTE events.
Assuntos
Classificação Internacional de Doenças/estatística & dados numéricos , Complicações Pós-Operatórias/epidemiologia , Indicadores de Qualidade em Assistência à Saúde/estatística & dados numéricos , United States Agency for Healthcare Research and Quality/estatística & dados numéricos , Tromboembolia Venosa/epidemiologia , Adulto , Fatores Etários , Idoso , Feminino , Humanos , Masculino , Pessoa de Meia-Idade , Reprodutibilidade dos Testes , Estudos Retrospectivos , Fatores Sexuais , Estados UnidosRESUMO
OBJECTIVE: To examine the association between the scope of quality improvement (QI) implementation in hospitals and hospital performance on selected indicators of clinical quality. DATA SOURCES: Secondary data from 1997 mailed survey of hospital QI practices, Medicare Inpatient Database, American Hospital Association's Annual Survey of Hospitals, the Bureau of Health Professions' Area Resource File, and two proprietary data sets compiled by Solucient Inc. containing data on managed care penetration and hospital financial performance. STUDY DESIGN: Cross-sectional study of 1,784 community hospitals to assess relationship between QI implementation approach and six hospital-level quality indicators. DATA COLLECTION/ABSTRACTION METHODS: Two-stage instrumental variables estimation in which predicted values (instruments) of four QI scope variables and control (exogenous) variables used to estimate hospital-level quality indicators. PRINCIPAL FINDINGS: Involvement by multiple hospital units in QI effort is associated with worse values on hospital-level quality indicators. Percentage of hospital staff and percentage of senior managers participating in formally organized QI teams are associated with better values on quality indicators. Percentage of physicians participating in QI teams is not associated with better values on the hospital-level quality indicators studied. CONCLUSIONS: Results supported the proposition that the scope of QI implementation in hospitals is significantly associated with hospital-level quality indicators. However, the direction of the association varied across different measures of QI implementation scope.
Assuntos
Pesquisa sobre Serviços de Saúde/métodos , Hospitais Comunitários/organização & administração , Garantia da Qualidade dos Cuidados de Saúde/organização & administração , Indicadores de Qualidade em Assistência à Saúde , Estudos Transversais , Humanos , Equipe de Assistência ao Paciente , Recursos Humanos em HospitalRESUMO
BACKGROUND: The National Quality Forum previously approved a quality indicator for mortality after congenital heart surgery developed by the Agency for Healthcare Research and Quality (AHRQ). Several parameters of the validated Risk Adjustment for Congenital Heart Surgery (RACHS-1) method were included, but others differed. As part of the National Quality Forum endorsement maintenance process, developers were asked to harmonize the 2 methodologies. METHODS AND RESULTS: Parameters that were identical between the 2 methods were retained. AHRQ's Healthcare Cost and Utilization Project State Inpatient Databases (SID) 2008 were used to select optimal parameters where differences existed, with a goal to maximize model performance and face validity. Inclusion criteria were not changed and included all discharges for patients <18 years with International Classification of Diseases, Ninth Revision, Clinical Modification procedure codes for congenital heart surgery or nonspecific heart surgery combined with congenital heart disease diagnosis codes. The final model includes procedure risk group, age (0-28 days, 29-90 days, 91-364 days, 1-17 years), low birth weight (500-2499 g), other congenital anomalies (Clinical Classifications Software 217, except for 758.xx), multiple procedures, and transfer-in status. Among 17 945 eligible cases in the SID 2008, the c statistic for model performance was 0.82. In the SID 2013 validation data set, the c statistic was 0.82. Risk-adjusted mortality rates by center ranged from 0.9% to 4.1% (5th-95th percentile). CONCLUSIONS: Congenital heart surgery programs can now obtain national benchmarking reports by applying AHRQ Quality Indicator software to hospital administrative data, based on the harmonized RACHS-1 method, with high discrimination and face validity.
Assuntos
Procedimentos Cirúrgicos Cardíacos , Cardiopatias Congênitas/cirurgia , Mortalidade Hospitalar , Indicadores de Qualidade em Assistência à Saúde , Adolescente , Fatores Etários , Benchmarking , Criança , Pré-Escolar , Feminino , Humanos , Lactente , Recém-Nascido de Baixo Peso , Recém-Nascido , Modelos Logísticos , Masculino , Mortalidade , Medição de Risco , Estados Unidos , United States Agency for Healthcare Research and QualityRESUMO
Measures based on routinely collected data would be useful to examine the epidemiology of patient safety. Extending previous work, we established the face and consensual validity of twenty Patient Safety Indicators (PSIs). We generated a national profile of patient safety by applying these PSIs to the HCUP Nationwide Inpatient Sample. The incidence of most nonobstetric PSIs increased with age and was higher among African Americans than among whites. The adjusted incidence of most PSIs was highest at urban teaching hospitals. The PSIs may be used in AHRQ's National Quality Report, while providers may use them to screen for preventable complications, target opportunities for improvement, and benchmark performance.
Assuntos
Hospitais/estatística & dados numéricos , Hospitais/normas , Doença Iatrogênica/epidemiologia , Erros Médicos/estatística & dados numéricos , Indicadores de Qualidade em Assistência à Saúde , Gestão da Segurança/estatística & dados numéricos , Pesquisas sobre Atenção à Saúde , Hospitais/classificação , Humanos , Incidência , Pacientes Internados/classificação , Pacientes Internados/estatística & dados numéricos , Classificação Internacional de Doenças , Erros Médicos/classificação , Medição de Risco , Gestão da Segurança/classificação , Estados Unidos/epidemiologiaRESUMO
BACKGROUND: Patient Safety Indicator (PSI) 9, "postoperative hemorrhage or hematoma" (PHH), of the US Agency for Healthcare Research and Quality has been considered for public quality of care reporting. We sought to evaluate its performance in detecting true complications. METHODS: We conducted a retrospective, cross-sectional study of hospitalizations that met PSI 9 eligibility criteria. We sampled records flagged positive and negative by PSI 9 from a diverse set of 31 hospitals between February 2006, and June 2009. Trained abstractors reviewed medical records using standard instruments. We determined the sensitivity, specificity, and positive (PPV) and negative (NPV) predictive values of the indicator. RESULTS: Of 181 analyzable records flagged by PSI 9, 168 (93%; weighted PPV, 95% [95% confidence interval (CI), 90-98%]) involved an accurately coded event, but only 126 (70%; weighted PPV, 78% [95% CI, 58-90%]) represented true PHH. Thirty-two false positives involved only intraoperative hemorrhage. Among true positives, hypotension occurred in 28% and death attributed to the PHH in 4%. Thirty-two of 281 records flagged negative by PSI 9 (but enriched with questionably negative records) represented true PHH. The indicator's sensitivity was 42% (95% CI, 23-64%), specificity 99.9% (95% CI, 99.8-100%), and NPV 99.7% (95% CI, 99.0-99.9%). Modifying the indicator to include additional procedure codes improved both sensitivity (85% [95% CI, 67-94%]) and PPV (76% [95% CI, 60-88%]). CONCLUSION: PSI 9 holds promise in detecting serious, possibly preventable complications. The indicator might be improved by specification of the 998.11 hemorrhage code to exclude purely intraoperative events and addition of procedure codes to the indicator's numerator criteria.
Assuntos
Hematoma/epidemiologia , Segurança do Paciente/estatística & dados numéricos , Hemorragia Pós-Operatória/epidemiologia , Indicadores de Qualidade em Assistência à Saúde/estatística & dados numéricos , Adulto , Idoso , Estudos Transversais , Feminino , Humanos , Masculino , Pessoa de Meia-Idade , Avaliação de Resultados em Cuidados de Saúde , Projetos Piloto , Indicadores de Qualidade em Assistência à Saúde/classificação , Estudos Retrospectivos , Estados Unidos/epidemiologiaRESUMO
OBJECTIVE: To test the implementation of a novel structured panel process in the evaluation of quality indicators. DATA SOURCE: National panel of 64 clinicians rating usefulness of indicator applications in 2008-2009. STUDY DESIGN: Hybrid panel combined Delphi Group and Nominal Group (NG) techniques to evaluate 81 indicator applications. PRINCIPAL FINDINGS: The Delphi Group and NG rated 56 percent of indicator applications similarly. Group assignment (Delphi versus Nominal) was not significantly associated with mean ratings, but specialty and research interests of panelists, and indicator factors such as denominator level and proposed use were. Rating distributions narrowed significantly in 20.8 percent of applications between review rounds. CONCLUSIONS: The hybrid panel process facilitated information exchange and tightened rating distributions. Future assessments of this method might include a control panel.
Assuntos
Técnica Delphi , Indicadores de Qualidade em Assistência à Saúde , Consenso , Feminino , Humanos , Masculino , Estados Unidos , United States Agency for Healthcare Research and QualityRESUMO
As part of the Agency for Healthcare Research and Quality Patient Safety Indicator (PSI) Validation Pilot Project, we evaluated the criterion validity of PSI 7. At the time of this study, PSI 7 was entitled "Selected Infections Due to Medical Care" and targeted catheter-related infections and inflammatory reactions. We conducted a retrospective cross-sectional study of 23 volunteer U.S. hospitals, where trained abstractors reviewed a sample of records that met PSI 7 criteria from October 1, 2005 to March 31, 2007. Of the 191 cases that met PSI 7 criteria, 104 (positive predictive value = 54%, 95% confidence interval: 40-69%) represented true infections. Of these cases, 77 (74%) were associated with central venous catheters, 15 (15%) were associated with peripheral intravenous (n=13) and or or arterial catheters (n=6), and 12 (11%) were associated with unknown catheters. Of the 87 (46%) false-positive cases, 41 (47%) did not have a qualifying infection identified by the abstractor, 38 (44%) had an infection present on admission, and 8 (9%) had an exclusionary diagnosis. PSI 7 has a low positive predictive value compared with other PSIs recently studied. Present on admission diagnoses and improved coding for infections related to central venous catheters (implemented October 2007) may improve validity.
Assuntos
Infecções Relacionadas a Cateter/epidemiologia , Cateterismo Venoso Central/efeitos adversos , Cateterismo Periférico/efeitos adversos , Indicadores de Qualidade em Assistência à Saúde , Intervalos de Confiança , Estudos Transversais , Reações Falso-Positivas , Feminino , Humanos , Masculino , Pessoa de Meia-Idade , Projetos Piloto , Valor Preditivo dos Testes , Estudos Retrospectivos , Estados Unidos/epidemiologia , United States Agency for Healthcare Research and QualityRESUMO
Hospital administrative data are being used to identify hospitals with hospital-acquired complications such as iatrogenic pneumothorax. This was a retrospective cross-sectional study of hospitalization records to estimate the positive predictive value (PPV) of the Agency for Healthcare Research and Quality (AHRQ) Patient Safety Indicator for iatrogenic pneumothorax. A probability sample of inpatient medical records from 2006 to 2007 was reviewed in a national sample of 28 volunteer hospitals. Among the 200 flagged cases, the PPV was 78% (95% confidence interval = 73%-82%). False positive cases were mostly a result of exclusionary conditions (11%) and pneumothoraxes that were present on admission (7%). About 44% of events followed attempted central venous catheter (CVC) placement. Of the 69 CVC-associated events, only 5 occurred with ultrasound guidance. AHRQ's iatrogenic pneumothorax indicator can serve in quality of care improvement. At least 1725 hospital-acquired pneumothoraxes could have been prevented in 2004 through universal use of ultrasound guidance during internal jugular cannulation.
Assuntos
Doença Iatrogênica/epidemiologia , Classificação Internacional de Doenças/estatística & dados numéricos , Prontuários Médicos/estatística & dados numéricos , Pneumotórax/epidemiologia , Indicadores de Qualidade em Assistência à Saúde , Cateterismo Venoso Central/efeitos adversos , Cateterismo Venoso Central/estatística & dados numéricos , Causalidade , Estudos Transversais , Humanos , Incidência , Pneumotórax/classificação , Pneumotórax/etiologia , Reprodutibilidade dos Testes , Estudos Retrospectivos , Medição de Risco , Procedimentos Cirúrgicos Torácicos/efeitos adversos , Procedimentos Cirúrgicos Torácicos/estatística & dados numéricos , Estados Unidos/epidemiologia , United States Agency for Healthcare Research and QualityRESUMO
OBJECTIVES: With >6 million hospital stays, costing almost $50 billion annually, hospitalized children represent an important population for which most inpatient quality indicators are not applicable. Our aim was to develop indicators using inpatient administrative data to assess aspects of the quality of inpatient pediatric care and access to quality outpatient care. METHODS: We adapted the Agency for Healthcare Research and Quality quality indicators, a publicly available set of measurement tools refined previously by our team, for a pediatric population. We systematically reviewed the literature for evidence regarding coding and construct validity specific to children. We then convened 4 expert panels to review and discuss the evidence and asked them to rate each indicator through a 2-stage modified Delphi process. From the 2000 and 2003 Agency for Healthcare Research and Quality Healthcare Cost and Utilization Project Kids' Inpatient Database, we generated national estimates for provider level indicators and for area level indicators. RESULTS: Panelists recommended 18 indicators for inclusion in the pediatric quality indicator set based on overall usefulness for quality improvement efforts. The indicators included 13 hospital-level indicators, including 11 based on complications, 1 based on mortality, and 1 based on volume, as well as 5 area-level potentially preventable hospitalization indicators. National rates for all 18 of the indicators varied minimally between years. Rates in high-risk strata are notably higher than in the overall groups: in 2003 the decubitus ulcer pediatric quality indicator rate was 3.12 per 1000, whereas patients with limited mobility experienced a rate of 22.83. Trends in rates by age varied across pediatric quality indicators: short-term complications of diabetes increased with age, whereas admissions for gastroenteritis decreased with age. CONCLUSIONS: Tracking potentially preventable complications and hospitalizations has the potential to help prioritize quality improvement efforts at both local and national levels, although additional validation research is needed to confirm the accuracy of coding.