Two-State Collaborative Study of a Multifaceted Intervention to Decrease Ventilator-Associated Events.

OBJECTIVES: Ventilator-associated events are associated with increased mortality, prolonged mechanical ventilation, and longer ICU stay. Given strong national interest in improving ventilated patient care, the National Institute of Health and Agency for Healthcare Research and Quality funded a two-state collaborative to reduce ventilator-associated events. We describe the collaborative's impact on ventilator-associated event rates in 56 ICUs. DESIGN: Longitudinal quasi-experimental study. SETTING: Fifty-six ICUs at 38 hospitals in Maryland and Pennsylvania from October 2012 to March 2015. INTERVENTIONS: We organized a multifaceted intervention to improve adherence with evidence-based practices, unit teamwork, and safety culture. Evidence-based interventions promoted by the collaborative included head-of-bed elevation, use of subglottic secretion drainage endotracheal tubes, oral care, chlorhexidine mouth care, and daily spontaneous awakening and breathing trials. Each unit established a multidisciplinary quality improvement team. We coached teams to establish comprehensive unit-based safety programs through monthly teleconferences. Data were collected on rounds using a common tool and entered into a Web-based portal. MEASUREMENTS AND RESULTS: ICUs reported 69,417 ventilated patient-days of intervention compliance observations and 1,022 unit-months of ventilator-associated event data. Compliance with all evidence-based interventions improved over the course of the collaborative. The quarterly mean ventilator-associated event rate significantly decreased from 7.34 to 4.58 cases per 1,000 ventilator-days after 24 months of implementation (p = 0.007). During the same time period, infection-related ventilator-associated complication and possible and probable ventilator-associated pneumonia rates decreased from 3.15 to 1.56 and 1.41 to 0.31 cases per 1,000 ventilator-days (p = 0.018, p = 0.012), respectively. CONCLUSIONS: A multifaceted intervention was associated with improved compliance with evidence-based interventions and decreases in ventilator-associated event, infection-related ventilator-associated complication, and probable ventilator-associated pneumonia. Our study is the largest to date affirming that best practices can prevent ventilator-associated events.

Clinician advice to quit smoking among seniors.

OBJECTIVE: Little smoking research in the past 20years includes persons 50 and older; herein we describe patterns of clinician cessation advice to US seniors, including variation by Medicare beneficiary characteristics. METHOD: In 2012-4, we analyzed 2010 Consumer Assessment of Healthcare Providers and Systems (CAHPS) survey data from Medicare beneficiaries over age 64 (n=346,674). We estimated smoking rates and the proportion of smokers whose clinicians encouraged cessation. RESULTS: 12% of male and 8% of female respondents aged 65 and older smoke. The rate decreases with age (14% of 65-69, 3% of 85+) and education (12-15% with no high school degree, 5-6% with BA+). Rates are highest among American Indian/Alaskan Native (16%), multiracial (14%), and African-American (13%) seniors, and in the Southeast (14%). Only 51% of smokers say they receive cessation advice "always" or "usually" at doctor visits, with advice more often given to the young, those in low-smoking regions, Asians, and women. For all results cited p<0.05. CONCLUSIONS: Smoking cessation advice to seniors is variable. Providers may focus on groups or areas in which smoking is less common or when they are most comfortable giving advice. More consistent interventions are needed, including cessation advice from clinicians.

Measurement of implementation components and contextual factors in a two-state healthcare quality initiative to reduce ventilator-associated pneumonia.

OBJECTIVE: To develop and field test an implementation assessment tool for assessing progress of hospital units in implementing improvements for the prevention of ventilator-associated pneumonia (VAP) in a two-state collaborative, including data on actions implemented by participating teams and contextual factors that may influence their efforts. Using the data collected, learn how implementation actions can be improved and analyze effects of implementation progress on outcome measures. DESIGN: We developed the tool as an interview protocol that included quantitative and qualitative items addressing actions on the Comprehensive Unit-based Safety Program (CUSP) and clinical interventions for use in guiding data collection via telephone interviews. SETTING: We conducted interviews with leaders of improvement teams from units participating in the two-state VAP prevention initiative. METHODS: We collected data from 43 hospital units as they implemented actions for the VAP initiative and performed descriptive analyzes of the data with comparisons across the 2 states. RESULTS: Early in the VAP prevention initiative, most units had made only moderate progress overall in using many of the CUSP actions known to support their improvement processes. For contextual factors, a relatively small number of barriers were found to have important negative effects on implementation progress (in particular, barriers related to workload and time issues). We modified coaching provided to the unit teams to reinforce training in weak spots that the interviews identified. CONCLUSION: These assessments provided important new knowledge regarding the implementation science of quality improvement projects, including feedback during implementation, and give a better understanding of which factors most affect implementation.

A retrospective evaluation of the Perfecting Patient Care University training program for health care organizations.

This study evaluated how the Perfecting Patient Care (PPC) University, a quality improvement (QI) training program for health care leaders and clinicians, affected the ability of organizations to improve the health care they provide. This training program teaches improvement methods based on Lean concepts and principles of the Toyota Production System and is offered in several formats. A retrospective evaluation was performed that gathered data on training, other process factors, and outcomes after staff completed the PPC training. A majority of respondents reported gaining QI competencies and cultural achievements from the training. Organizations had high average scores for the success measures of "outcomes improved" and "sustainable monitoring" but lower scores for diffusion of QI efforts. Total training dosage was significantly associated with the measures of QI success. This evaluation provides evidence that organizations gained the PPC competencies and cultural achievements and that training dosage is a driver of QI success.

Specialties differ in which aspects of doctor communication predict overall physician ratings.

BACKGROUND: Effective doctor communication is critical to positive doctor-patient relationships and predicts better health outcomes. Doctor communication is the strongest predictor of patient ratings of doctors, but the most important aspects of communication may vary by specialty. OBJECTIVE: To determine the importance of five aspects of doctor communication to overall physician ratings by specialty. DESIGN: For each of 28 specialties, we calculated partial correlations of five communication items with a 0-10 overall physician rating, controlling for patient demographics. PATIENTS: Consumer Assessment of Healthcare Providers and Systems Clinician and Group (CG-CAHPS®) 12-month Survey data collected 2005-2009 from 58,251 adults at a 534-physician medical group. MAIN MEASURES: CG-CAHPS includes a 0 ("Worst physician possible") to 10 ("Best physician possible") overall physician rating. Five doctor communication items assess how often the physician: explains things; listens carefully; gives easy-to-understand instructions; shows respect; and spends enough time. KEY RESULTS: Physician showing respect was the most important aspect of communication for 23/28 specialties, with a mean partial correlation (0.27, ranging from 0.07 to 0.44 across specialties) that accounted for more than four times as much variance in the overall physician rating as any other communication item. Three of five communication items varied significantly across specialties in their associations with the overall rating (p < 0.05). CONCLUSIONS: All patients valued respectful treatment; the importance of other aspects of communication varied significantly by specialty. Quality improvement efforts by all specialties should emphasize physicians showing respect to patients, and each specialty should also target other aspects of communication that matter most to their patients. The results have implications for improving provider quality improvement and incentive programs and the reporting of CAHPS data to patients. Specialists make important contributions to coordinated patient care, and thus customized approaches to measurement, reporting, and quality improvement efforts are important.

Care experiences of managed care Medicare enrollees near the end of life.

OBJECTIVES: To compare reports about care experiences of individuals who died within 1 year of survey with reports of those who did not. DESIGN: Medicare Advantage (MA) Consumer Assessment of Healthcare Providers and Systems (CAHPS) surveys asked about care experiences. Survey completion dates were linked to Social Security Administration death records to identify enrollees dying within 1 year of survey completion. Propensity-score weighting combined with regression-based case-mix adjustment was used to compare these individuals' experiences with experiences of those who were alive 1 year later. SETTING: Nationally representative sample of MA enrollees. PARTICIPANTS: Four hundred two thousand five hundred ninety-three MA enrollees responding to 2008 and 2009 CAHPS Surveys. MEASUREMENTS: Outcomes were five care ratings (plan, prescription drug coverage, doctor, specialists, care) and five composite measures of care (getting needed care, getting care quickly, doctor communication, getting drugs, getting drug information). Analyses were adjusted for age, sex, race and ethnicity, education, Medicaid status, geographic region, and several health status measures. RESULTS: Twelve thousand one hundred two enrollees (3%) died within 1 year of survey completion (near-end-of-life group). Those enrollees reported slightly better experiences than other enrollees with respect to getting care quickly (+2%, P < .001) and gave slightly higher ratings for their plans (+1%, P = .02) and prescription drug coverage (+1%, P < .001). There were no measures of participant experience for which the near-end-of-life group reported worse experiences than other enrollees. CONCLUSION: Contrary to analyses based on retrospective reports from surviving relatives after an individual's death, MA enrollees' reports about care within 1 year of death were as good as or better than reports of other MA enrollees. Future research might investigate whether results are similar in other Medicare populations.

Experiences of care among Medicare beneficiaries with ESRD: Medicare Consumer Assessment of Healthcare Providers and Systems (CAHPS) survey results.

BACKGROUND: Patients with end-stage renal disease (ESRD) have special health needs; little is known about their care experiences. STUDY DESIGN: Secondary analysis of 2009-2010 Medicare Consumer Assessment of Healthcare Providers and Systems (CAHPS) data, using representative random samples of Medicare beneficiaries. Description of Medicare beneficiaries with ESRD and investigation of differences in patient experiences by sociodemographic characteristics and coverage type. SETTING & PARTICIPANTS: Data were collected from 823,564 Medicare beneficiaries (3,794 with ESRD) as part of the Medicare CAHPS survey, administered by mail with telephone follow-up of nonrespondents. PREDICTOR: ESRD status, age, education, self-reported general and mental health status, race/ethnicity, sex, Medicare coverage type, state of residence, and other demographic measures. OUTCOMES: 6 composite measures of patient experience in 4 care domains (access to care, physician communication, customer service, and access to prescription drugs and drug information) and 4 ratings (overall care, personal physician, specialist physician, and prescription drug plan). RESULTS: Patients with ESRD reported better care experiences than non-ESRD beneficiaries for 7 of 10 measures (P < 0.05) after adjustment for patient characteristics, geography, and coverage type, although to only a small extent (adjusted mean difference, <3 points [scale, 0-100]). Black patients with ESRD and less educated patients were more likely than other patients with ESRD to report poor experiences. LIMITATIONS: Inability to distinguish patient experiences of care for different treatment modalities. CONCLUSIONS: On average, beneficiaries with ESRD report patient experiences that are at least as positive as non-ESRD beneficiaries. However, black and less educated patients with ESRD reported worse experiences than other ESRD patients. Stratified reporting of patient experience by race/ethnicity or education in patients with ESRD can be used to monitor this disparity. Physician choice and confidence and trust in physicians may be particularly important for patients with ESRD.

CMS Innovation Center Health Care Innovation Awards: Evaluation Plan.

The Center for Medicare and Medicaid Innovation within the Centers for Medicare & Medicaid Services (CMS) has funded 108 Health Care Innovation Awards, funded through the Affordable Care Act, for applicants who proposed compelling new models of service delivery or payment improvements that promise to deliver better health, better health care, and lower costs through improved quality of care for Medicare, Medicaid, and Children's Health Insurance Program enrollees. CMS is also interested in learning how new models would affect subpopulations of beneficiaries (e.g., those eligible for Medicare and Medicaid and complex patients) who have unique characteristics or health care needs that could be related to poor outcomes. In addition, the initiative seeks to identify new models of workforce development and deployment, as well as models that can be rapidly deployed and have the promise of sustainability. This article describes a strategy for evaluating the results. The goal for the evaluation design process is to create standardized approaches for answering key questions that can be customized to similar groups of awardees and that allow for rapid and comparable assessment across awardees. The evaluation plan envisions that data collection and analysis will be carried out on three levels: at the level of the individual awardee, at the level of the awardee grouping, and as a summary evaluation that includes all awardees. Key dimensions for the evaluation framework include implementation effectiveness, program effectiveness, workforce issues, impact on priority populations, and context. The ultimate goal is to identify strategies that can be employed widely to lower cost while improving care.

Measuring success for health care quality improvement interventions.

BACKGROUND: The lack of a standard measure of quality improvement (QI) success and the use of subjective or self-reported measures of QI success has constrained efforts to formally evaluate QI programs and to understand how the various contextual factors impact QI success. OBJECTIVES: The objective of this study was to assess how best to measure "QI success" by comparing self-reported and externally rated measures of QI success. RESEARCH DESIGN: We performed a retrospective evaluation that analyzed data on different measures of QI success for organizations after their staff completed the QI training. SUBJECTS: The sample included 30 organizations whose staff had received QI training during 2006-2008, and who had used this training to carry out at least some subsequent QI initiative in their organizations. MEASURES: We developed 2 measures of self-reported QI success based on survey responses and 4 externally rated measures of QI success based on outcome data provided by the participating organizations in addition to qualitative data generated from the interviews. RESULTS: We found some variation in the mean scores of the different QI success measures and only moderate to small correlations between the self-report and externally rated QI measures. CONCLUSIONS: This study confirms that there are important differences between self-reported and externally rated measures of QI success and provides researchers with a methodology and criteria to externally rate measures of QI success.

Multiple patient safety events within a single hospitalization: a national profile in US hospitals.

The objective was to examine co-occurrence of iatrogenic events in US hospitals. Using Agency for Healthcare Research and Quality patient safety indicators (PSIs), the authors defined multiple patient safety events (MPSEs) as the occurrence of multiple PSIs during a single hospitalization. The National Inpatient Sample was analyzed to estimate the national prevalence of MPSEs, risk factors for MPSEs, and the average length of stay and average hospital charges associated with MPSEs. MPSEs occurred in approximately 1 in every 1000 hospitalizations, affecting more than 30 000 patients in 2004. Significant risk factors for MPSEs include age, black race, Medicare coverage, and treatment at urban teaching hospitals. Compared with all admissions, the average length of stay for MPSE admissions was 4 times longer, and the average charge for MPSE admissions was 8 times greater. Despite the low prevalence, MPSEs affect large numbers of hospital patients in America. MPSEs have distinct characteristics and are far more resource intensive than hospital admissions generally.

How event reporting by US hospitals has changed from 2005 to 2009.

CONTEXT: Information is needed on the performance of hospitals' adverse-event reporting systems and the effects of national patient-safety initiatives, including the Patient Safety and Quality Improvement Act (PSQIA) of 2005. Results are presented of a 2009 survey of a sample of non-federal US hospitals and changes between 2005 and 2009 are examined. METHODS: The Adverse Event Reporting System survey was fielded in 2005 and 2009 using a mixed-mode design with stratified random samples of non-federal US hospitals; risk managers were respondents. Response rates were 81% in 2005 and 79% in 2009. RESULTS: Virtually all hospitals reported they had centralised adverse-event-reporting systems. However, scores on four performance indexes suggested that hospitals have not effectively implemented key components of reporting systems. Average index scores improved somewhat between 2005 and 2009 for supportive environment (0.7 increase; p<0.05) and types of staff reporting (0.08 increase; p<0.001). Average scores did not change for timely distribution of event reports or discussion with key departments and committees. Some within-hospital inconsistencies in responses between 2005 and 2009 were found. These self-reported responses may be optimistic assessments of hospital performance. CONCLUSIONS: The 2009 survey confirmed improvement needs identified by the 2005 survey for hospitals' event reporting processes, while finding signs of progress. Optimising the use of surveys to assess the effects of national patient-safety initiatives such as PSQIA will require decreasing within-hospital variations in reporting rates.

Depression and the health care experiences of Medicare beneficiaries.

OBJECTIVES: To compare health care experiences of Medicare beneficiaries with and without symptoms of depression and investigate the role of patient confidence in shaping these experiences. DATA SOURCES: Data came from the 2009 CAHPS Medicare 4.0 Fee-for-Service (FFS) Survey, which was fielded to a national probability sample of 298,492 FFS Medicare beneficiaries. STUDY DESIGN: Linear regression was used to model associations of depression with four global ratings and three composite measures of health care and to test whether beneficiaries' confidence in their ability to recognize the need for care mediates these associations. PRINCIPAL FINDINGS: Beneficiaries with depressive symptoms reported worse experiences with care across the full range of patient experience covered by the CAHPS survey. Depressive symptoms were associated with decreased patient confidence and decreased confidence was in turn associated with poorer reports of care. CONCLUSIONS: Our study highlights depressive symptoms as a risk factor for poorer experiences of health care and highlights depressed patients' confidence in recognizing their need for care and for designing programs to improve the health care of this population.

Understanding variations in Medicare Consumer Assessment of Health Care Providers and Systems scores: California as an example.

OBJECTIVE: To understand reasons why California has lower Consumer Assessment of Healthcare Providers and Systems (CAHPS) scores than the rest of the country, including differing patterns of CAHPS scores between Medicare Advantage (MA) and fee-for-service, effects of additional demographic characteristics of beneficiaries, and variation across MA plans within California. STUDY DESIGN/DATA COLLECTION: Using 2008 CAHPS survey data for fee-for-service Medicare beneficiaries and MA members, we compared mean case mix adjusted Medicare CAHPS scores for California and the remainder of the nation. PRINCIPAL FINDINGS: California fee-for-service Medicare had lower scores than non-California fee-for-service on 11 of 14 CAHPS measures; California MA had lower scores only for physician services measures and higher scores for other measures. Adding race/ethnicity and urbanity to risk adjustment improved California standing for all measures in both MA and fee-for-service. Within the MA plans, one large plan accounted for the positive performance in California MA; other California plans performed below national averages. CONCLUSIONS: This study shows that the mix of fee-for-service and MA enrollees, demographic characteristics of populations, and plan-specific factors can all play a role in observed regional variations. Anticipating value-based payments, further study of successful MA plans could generate lessons for enhancing patient experience for the Medicare population.

Advancing the science of patient safety.

Despite a decade's worth of effort, patient safety has improved slowly, in part because of the limited evidence base for the development and widespread dissemination of successful patient safety practices. The Agency for Healthcare Research and Quality sponsored an international group of experts in patient safety and evaluation methods to develop criteria to improve the design, evaluation, and reporting of practice research in patient safety. This article reports the findings and recommendations of this group, which include greater use of theory and logic models, more detailed descriptions of interventions and their implementation, enhanced explanation of desired and unintended outcomes, and better description and measurement of context and of how context influences interventions. Using these criteria and measuring and reporting contexts will improve the science of patient safety.

"Canary measures" among the AHRQ patient safety indicators.

Despite the fact that the Agency for Healthcare Research and Quality patient safety indicators (PSIs) are useful for evaluating adverse events in hospitals, the 20 PSIs make it difficult to summarize broad trends across indicators. This study aims to identify 1 or more measures among the PSIs best suited for a quick index of general patient safety status. The 1997-2002 Nationwide Inpatient Sample was used to calculate the hospital-level risk-adjusted rates of the PSIs. Both pairwise correlation analyses and multivariate analyses showed that PSI #7 (Selected infections due to medical care) is significantly correlated with most of the other PSIs. The results are robust even when (1) the pairwise and multivariate analyses are rerun on a reduced data set that excludes admissions with multiple patient safety events, and (2) Bonferroni-adjusted P values are calculated. The study concludes that PSI #7 appears to be a candidate for a quick index of patient safety status.

Using health information technology-related performance measures and tools to improve chronic care.

BACKGROUND: The American Medical Association led a collaborative initiative to explore opportunities for improving the quality of outpatient chronic care through the use of nationally endorsed clinical performance measures and tools. The measures and tools focused on adult diabetes, major depressive disorder, chronic stable coronary artery disease, heart failure, hypertension, and asthma. METHODS: The RAND Corporation conducted an independent, formative assessment of the initiative's four pilot activities using the Context-Input-Process-Product evaluation model. RESULTS: Pilots 1 and 2 demonstrated the feasibility and value of implementing performance measures and tools in practices with electronic health information systems, while highlighting the difficulty of using them in practices with paper-based systems and in community-based models, where multiple stakeholders are expected to share patient data. Pilot 3 illustrated the usefulness of validating performance measures before their use for internal quality improvement or external reporting. Pilot 4 documented the challenges involved in exporting clinical performance data from a physician practice to external entities for multiple potential uses. DISCUSSION: Improving the quality of chronic care through clinical performance measurement, data aggregation, and reporting will require expanded use of clinical performance measures for both internal quality improvement and pay-for-performance; integrating electronic health records (EHRs) or electronic-based registries into more physician offices; more accurate measurement and documentation of diagnoses and care procedures; EHR products that make it easier to capture certain types of information; and simplified, standardized processes for performance data extraction and exporting.

Evaluation of the AHRQ patient safety initiative: framework and approach.

OBJECTIVE: Describe the evaluation performed of the patient safety initiative operated by the Agency for Healthcare Research and Quality (AHRQ). AHRQ PATIENT SAFETY INITIATIVE When patient safety became a national priority in 2000, Congress charged and funded AHRQ to improve health care safety. Over the next 6 years, AHRQ funded more than 300 research projects and other activities, addressing diverse patient safety issues and practices. THE PATIENT SAFETY EVALUATION: AHRQ contracted with RAND in 2002 to perform a 4-year evaluation of the initiative, which was completed in 2006. This formative evaluation used the CIPP program evaluation model, which emphasizes multiple stakeholders' interests (e.g., patients, providers, funded researchers). We monitored the progress of the patient safety initiative and provided AHRQ annual feedback that assessed each year's activities, identifying issues and offering suggestions for actions by AHRQ. Given the size and complexity of the initiative, the evaluation needed to examine key individual components and synthesize results across them, and it also had to be responsive to changes in the initiative over time. We used a conceptual framework to bring together the disparate pieces to synthesize overall findings. The remaining articles in this issue describe selected results from this evaluation.

Assessment of contributions to patient safety knowledge by the Agency for Healthcare Research and Quality-funded patient safety projects.

OBJECTIVE: To characterize the activities of projects funded in Agency for Healthcare Research and Quality (AHRQ)'s patient safety portfolio and assess their aggregate potential to contribute to knowledge development. DATA SOURCES: Information abstracted from proposals for projects funded in AHRQ's patient safety portfolio, information on safety practices from the AHRQ Evidence Report on Patient Safety Practices, and products produced by the projects. STUDY DESIGN: This represented one part of the process evaluation conducted as part of a longitudinal evaluation based on the Context­Input­Process­Product model. PRINCIPAL FINDINGS: The 234 projects funded through AHRQ's patient safety portfolio examined a wide variety of patient safety issues and extended their work beyond the hospital setting to less studied parts of the health care system. Many of the projects implemented and tested practices for which the patient safety evidence report identified a need for additional evidence. The funded projects also generated a substantial body of new patient safety knowledge through a growing number of journal articles and other products. CONCLUSIONS: The projects funded in AHRQ's patient safety portfolio have the potential to make substantial contributions to the knowledge base on patient safety. The full value of this new knowledge remains to be confirmed through the synthesis of results