Measuring Access to Mental Health Services Among Primary Care Patients.

BACKGROUND: The lifetime risk of mental health disorders is almost 50% and, in any year, about 25% of the population have a psychiatric disorder. Many of those people are cared for in primary care settings. RESEARCH OBJECTIVE: Measure access to mental health services, such as getting counselling or prescription mental health medications, using new patient survey questions that can be added to Consumer Assessment of Healthcare Providers and Systems (CAHPS) surveys. STUDY DESIGN: Surveys were conducted with a stratified probability sample of patients receiving primary care services in a single state in 2018-2019. Medicaid and privately insured patients were surveyed by mail or telephone, respectively. RESULTS: Approximately 14% of sampled patients responded to a survey. More than 10% of privately insured respondents and about 20% of Medicaid respondents got or tried to get appointments for mental health care. About 15% of privately insured respondents and 11% of Medicaid respondents reported problems getting appointments with counselors. Only 8%-9% of respondents seeking mental health medicines reported problems getting appointments for prescriptions. A composite measure combining access to counselors and prescribers of mental health medicines evidenced adequate internal consistency reliability. Group level reliability estimates were low. CONCLUSIONS: Many respondents got or tried to get mental health services and a substantial number reported problems getting appointments or getting mental health prescriptions. The tested questions can be combined into an Access to Mental Health Care measure, which can be included in patient experience surveys for ambulatory care to monitor access to behavioral health care.

A Systematic Review of Strategies to Enhance Response Rates and Representativeness of Patient Experience Surveys.

BACKGROUND: Data from surveys of patient care experiences are a cornerstone of public reporting and pay-for-performance initiatives. Recently, increasing concerns have been raised about survey response rates and how to promote equity by ensuring that responses represent the perspectives of all patients. OBJECTIVE: Review evidence on survey administration strategies to improve response rates and representativeness of patient surveys. RESEARCH DESIGN: Systematic review adhering to the Preferred Reporting Items for Systematic reviews and Meta-Analyses guidelines. STUDY SELECTION: Forty peer-reviewed randomized experiments of administration protocols for patient experience surveys. RESULTS: Mail administration with telephone follow-up provides a median response rate benefit of 13% compared with mail-only or telephone-only. While surveys administered only by web typically result in lower response rates than those administered by mail or telephone (median difference in response rate: -21%, range: -44%, 0%), the limited evidence for a sequential web-mail-telephone mode suggests a potential response rate benefit over sequential mail-telephone (median: 4%, range: 2%, 5%). Telephone-only and sequential mixed modes including telephone may yield better representation across patient subgroups by age, insurance type, and race/ethnicity. Monetary incentives are associated with large increases in response rates (median increase: 12%, range: 7%, 20%). CONCLUSIONS: Sequential mixed-mode administration yields higher patient survey response rates than a single mode. Including telephone in sequential mixed-mode administration improves response among those with historically lower response rates; including web in mixed-mode administration may increase response at lower cost. Other promising strategies to improve response rates include in-person survey administration during hospital discharge, incentives, minimizing survey language complexity, and prenotification before survey administration.

How responding in Spanish affects CAHPS results.

BACKGROUND: The most widely used surveys for assessing patient health care experiences in the U.S. are the Consumer Assessment of Healthcare Providers and Systems (CAHPS) surveys. Studies examining the associations of language and ethnicity with responses to CAHPS surveys have yielded inconsistent findings. More research is needed to assess the effect of responding to CAHPS surveys in Spanish. METHODS: Subjects were patients who had received care at a study community health center in Connecticut within 6 or 12 months of being sent a CAHPS survey that asks about care experiences. The survey included four multi-item measures of care plus an overall rating of the provider. Sampled patients were mailed dual language (English and Spanish) cover letters and questionnaires. Those who did not respond after follow-up mailings were contacted by bilingual interviewers to complete the survey by telephone. We tested three hypotheses for any observed differences by ethnicity and language: 1. Spanish speakers are more likely than others to choose extreme response options. 2. The semantic meaning of the Spanish translation is not the same as the English version of the questions, resulting in Spanish speakers giving different answers because of meaning differences. 3. Spanish speakers have different expectations regarding their health care than those who answer in English. Analyses compared the answers on the survey measures for three groups: non-Hispanics answering in English, Hispanics answering in English, and Hispanics answering in Spanish. RESULTS: The overall response rate was 45%. After adjusting for differences in demographic characteristics and self-rated health, those answering in Spanish gave significantly more positive reports than the other two groups on three of the five measures, and higher than the non-Hispanic respondents on a fourth. CONCLUSIONS: Those answering in Spanish gave more positive reports of their medical experiences than Hispanics and non-Hispanics answering in English. Whether these results reflect different response tendencies, different standards for care, or better care experiences is a key issue in whether CAHPS responses in Spanish need adjustment to make them comparable to responses in English.

Comparing Web and Mail Protocols for Administering Hospital Consumer Assessment of Healthcare Providers and Systems Surveys.

OBJECTIVE: The objective of this study was to compare results of using web-based and mail (postal) Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) data collection protocols. RESEARCH DESIGN: Patients who had been hospitalized in a New England Hospital were surveyed about their hospital experience. Patients who provided email addresses were randomized to 1 of 3 data collection protocols: web-alone, web with postal mail follow-up, and postal mail only. Those who did not provide email addresses were surveyed using postal mail only. Analyses compared response rates, respondent characteristics, and patient-reported experiences. SUBJECTS: For an 8-week period, patients were discharged from the study hospital to home. MEASURES: Measures included response rates, characteristics of respondents, 6 composite measures of their patient experiences, and 2 ratings of the hospital. RESULTS: Response rates were significantly lower for the web-only protocol than the mail or combined protocols, and those who had not provided email addresses had lower response rates. Those over 65 were more likely than others to respond to all protocols, especially for the mail-only protocols. Respondents without email addresses were older, less educated, and reported worse health than those who provided email addresses. After adjusting for respondent differences, those in the combined protocol differed significantly from the mail (postal) only respondents on 2 measures of patient experience; those in the web-only protocol differed on one. Those not providing an email address differed from those who did on one measure. CONCLUSION: If web-based protocols are used for HCAHPS surveys, adjustments for a mode of data collection are needed to make results comparable.

Is Patient-Physician Gender Concordance Related to the Quality of Patient Care Experiences?

BACKGROUND: There is great interest in identifying factors that are related to positive patient experiences such as physician communication style. Documented gender-specific physician communication and patient behavior differences raise the question of whether gender concordant relationships (i.e., both the provider and patient share the same gender) might affect patient experiences. OBJECTIVE: Assess whether patient experiences are more positive in gender concordant primary care relationships. DESIGN: Statewide telephone surveys. Linear mixed regression models to estimate the association of CAHPS scores with patient gender and gender concordance. SUBJECTS: Two probability samples of primary care Medicaid patients in Connecticut in 2017 (5/17-7/17) and 2019 (7/19-10/19). MAIN MEASURES: Clinician and Group Consumer Assessment of Healthcare Providers and Systems (CG-CAHPS) survey augmented with questions about aspects of care most salient to PCMH-designated organizations and two questions to assess access to mental health services. KEY RESULTS: There were no significant effects of gender concordance and differences in experiences by patient gender were modest. CONCLUSIONS: This study did not support the suggestion that patient and physician gender and gender concordance have an important effect on patient experiences.

Effects of Leader Tactics on the Creativity, Implementation, and Evolution of Ideas to Improve Healthcare Delivery.

BACKGROUND: Slow progress in quality improvement (QI) has prompted calls to identify new QI ideas. Leaders guiding these efforts are advised to use evidence-based tactics, or specific approaches to address a goal, to promote clinician and staff engagement in the generation and implementation of QI ideas, but little evidence about effective tactics exists. OBJECTIVE: Examine the association between leader tactics and the creativity, implementation outcome, and evolution of QI ideas from clinicians and staff. DESIGN: Prospective panel analysis of 220 ideas generated by 12 leaders and teams (N = 72 members) from federally qualified community health practices in one center over 18 months. Measures were extracted from meeting minutes (note-taking by a member during meetings) and expert panel review. Multi-level models were used. MEASURES: Leader tactics, idea creativity, implementation outcome, evolution pathways, center, and idea-submitter characteristics. RESULTS: Leaders used one of four approaches: no tactic, meeting ground rules, team brainstorming, or reflection on team process. Implemented ideas evolved in three pathways: Plug and Play, Slow Burn, and Iterate and Generate. Compared with no leader tactic, meeting ground rules resulted in ideas not significantly different in creativity, implementation outcome, or evolution pathway. Brainstorming was associated with greater idea creativity, idea implementation, and ideas following a Plug and Play path (low member engagement and implementation over 2 months or less). Reflection on team process was associated with idea implementation (versus not), and ideas following an Iterate and Generate path (high member engagement and implementation over 3 months or more). CONCLUSIONS: Two tactics, brainstorming and reflection, are helpful depending on goals. Brainstorming may aide leaders seeking disruptive change via more creative, rapidly implemented ideas. Reflection on team process may aide leaders seeking high-engagement ideas that may be implemented slowly. Both tactics may help leaders cultivate dynamics that increase implementation of ideas that improve healthcare.

The How Matters: How Primary Care Provider Communication With Team Relates to Patients' Disease Management.

BACKGROUND: Investigating primary care provider (PCP)-team communication can provide insight into how colleagues work together to become high-functioning teams more able to address an increasingly complex set of tasks associated with chronic disease management. OBJECTIVE: To assess how PCP communication with their care team relates to patients' health. RESEARCH DESIGN: Longitudinal study of how 3 aspects of PCP-care team communication-participation, time spent listening, and uninterrupted speaking length-relate to disease management of patients with hypertension or diabetes, and the effect of these team communication behaviors on PCP-patient communication as a pathway by which this relationship might exist. We used multilevel regression models. SUBJECTS: Twenty-seven PCPs and 98 team members, and 18,067 patients with hypertension and 8354 patients with diabetes affiliated with a federally qualified health center with 12 practice sites. MEASURES: Primary data on communication collected using sociometric sensors worn by PCPs and team members, patient-PCP communication data collected with surveys, and patient health, PCP and patient characteristics extracted from electronic records. RESULTS: PCPs participated in 75% of care team conversations, spent 56% of conversation time listening, and had an average uninterrupted speaking length of 2.42 seconds. PCP participation, listening, and length of uninterrupted speaking time were associated with significantly higher odds that their patients had controlled hypertension and diabetes and improvements in disease control over time. PCP-patient communication mediates this relationship. CONCLUSIONS: PCP-team communication is associated with patient health management. How team members speak with one another may be as important as the content of their communication.

A quasi-experiment assessing the six-months effects of a nurse care coordination program on patient care experiences and clinician teamwork in community health centers.

BACKGROUND: Recognition that coordination among healthcare providers is associated with better quality of care and lower costs has increased interest in interventions designed to improve care coordination. One intervention is to add care coordination to nurses' role in a formal way. Little is known about effects of this approach, which tends to be pursued by small organizations and those in lower-resource settings. We assessed effects of this approach on care experiences of high-risk patients (those most in need of care coordination) and clinician teamwork during the first 6 months of use. METHODS: We conducted a quasi-experimental study using a clustered, controlled pre-post design. Changes in staff and patient experiences at six community health center practice locations that introduced the added-role approach for high-risk patients were compared to changes in six locations without the program in the same health system. In the pre-period (6 months before intervention training) and post-period (about 6 months after intervention launch, following 3 months of training), we surveyed clinical staff (N = 171) and program-qualifying patients (3007 pre-period; 2101 post-period, including 113 who were enrolled during the program's first 6 months). Difference-in-differences models examined study outcomes: patient reports about care experiences and clinician-reported teamwork. We assessed frequency of patient office visits to validate access and implementation, and contextual factors (training, resources, and compatibility with other work) that might explain results. RESULTS: Patient care experiences across all high-risk patients did not improve significantly (p > 0.05). They improved somewhat for program enrollees, 5% above baseline reports (p = 0.07). Staff-perceived teamwork did not change significantly (p = 0.12). Office visits increased significantly for enrolled patients (p < 0.001), affirming program implementation (greater accessing of care). Contextual factors were not reported as problematic, except that 41% of nurses reported incompatibility between care coordination and other job demands. Over 75% of nurses reported adequate training and resources. CONCLUSIONS: There were some positive effects of adding care coordination to nurses' role within 6 months of implementation, suggesting value in this improvement strategy. Addressing compatibility between coordination and other job demands is important when implementing this approach to coordination.

Differences in Hospitalizations Between Fee-for-Service and Medicare Advantage Beneficiaries.

BACKGROUND: Previous studies found lower hospitalization rates for enrollees in Medicare Advantage (MA) plans than for beneficiaries with fee-for-service (FFS) coverage. MA enrollment is increasing, especially for those newly eligible for Medicare, but little is known about how service use in FFS or MA differs for new beneficiaries. OBJECTIVE: To compare differences in rates of hospitalization between MA and FFS. RESEARCH DESIGN: A retrospective study of hospitalization among FFS and MA respondents to the Medicare Consumer Assessment of Healthcare Providers and Systems (MCAHPS) survey. Differences in hospitalization rates were assessed using multivariable logistic regression models that controlled for patient sociodemographic and health characteristics. Models included an interaction between age and coverage type to determine whether patterns of care were distinct for enrollees recently eligible for Medicare. STUDY POPULATION: In total, 259,335 respondents to the 2013 MCAHPS survey. RESULTS: In total, 14% of FFS and 12% of MA enrollees had ≥1 hospitalization in the 6 months before survey administration. Models adjusted for enrollee demographics found that MA enrollees had 0.81 the odds of being hospitalized relative to those with FFS coverage (95% confidence interval, 0.78-0.84). Differences between groups were substantially reduced and no longer statistically significant when they were fully adjusted (adjusted odds ratio 1.01, 95% confidence interval, 0.97-1.08). Models with interactions indicated no significant age differences in the MA/FFS hospitalization gap. CONCLUSION: Differences in hospital admissions between those with MA and FFS coverage appear to be primarily related to differences in health status.

International variations in trust in health care systems.

Public trust in health care systems has been measured in many countries, but there have been few studies of the intercountry variability in trust, or the degree to which such variability is because of population or structural characteristics. We used data from the health care survey conducted by the International Social Survey Program from 2011 to 2013 in 31 countries to assess whether intercountry variability was significantly greater than intracountry variability using general linear models in which country was treated as a fixed factor. We also assessed the extent to which intercountry variability was because of respondent and economic circumstances (gross national income per capita). Public trust in the health care system varied significantly across countries (P < .001), even after adjustment for 8 within-country predictors and gross national income per capita. One of the strongest predictors of trust was the respondents' most recent health care experience. Higher respondent education, urban residence, and a lower country's gross national income predicted less trust in the health care system. After countries with the 10% highest health expenditures per capita (United States) and the 10% lowest health care expenditures per capita (China and the Philippines) were removed, public trust in the health care system was positively associated with the remaining countries' health care expenditures per capita (Pearson correlation coefficient, 0.490; P = .008) and gross national income per capita (Pearson correlation coefficient, 0.495; P = .007). There is significant variation in public trust in health care across the countries studied. The intercountry differences are due, in part to economic circumstances.

Chronic Obstructive Pulmonary Disease Readmissions and Other Measures of Hospital Quality.

RATIONALE: The Centers for Medicare and Medicaid Services recently implemented financial penalties to reduce hospital readmissions for select conditions, including chronic obstructive pulmonary disease (COPD). Despite growing pressure to reduce COPD readmissions, it is unclear how COPD readmission rates are related to other measures of quality, which could inform efforts on common organizational factors that affect high-quality care. OBJECTIVES: To examine the association between COPD readmissions and other quality measures. METHODS: We analyzed data from the 2015 Centers for Medicare and Medicaid Services annual files, downloaded from the Hospital Compare website. We included 3,705 hospitals nationwide that had publically reported data on COPD readmissions. We compared COPD readmission rates to other risk-adjusted measures of quality, including readmission and mortality rates for other conditions, and patient reports about care experiences. MEASUREMENTS AND MAIN RESULTS: There were modest correlations between COPD readmission rates and readmission rates for other medical conditions, including heart failure (r = 0.39; P < 0.01), acute myocardial infarction (r = 0.30; P < 0.01), pneumonia (r = 0.38; P < 0.01), and stroke (r = 0.29; P < 0.01). In contrast, we found low correlations between COPD readmission rates and readmission rates for surgical conditions, as well as mortality rates for all measured conditions. There were significant correlations between COPD readmission rates and all patient experience measures. CONCLUSIONS: These findings suggest there may be common organizational factors that influence multiple disease-specific outcomes. As pay-for-performance programs focus attention on individual disease outcomes, hospitals may benefit from in-depth assessments of organizational factors that affect multiple aspects of hospital quality.

Evolving Concepts of Patient-Centered Care and the Assessment of Patient Care Experiences: Optimism and Opposition.

In his seminal work on health care quality, Avedis Donabedian noted that patient satisfaction was a key indicator of health care quality, and in the 1970s and 1980s a great deal of research explored the determinants of patient satisfaction. Subsequently, attention shifted toward assessing care experiences, and there is now a large body of evidence related to the reliability and validity of survey-based assessments of care. As the use of such surveys has increased, so too have concerns about the validity and uses of such surveys. The available research, however, indicates that such surveys are reliable, valid, correlated across individuals and settings with other quality indicators, and predictive of better outcomes. Patient experiences are now routinely measured, and substantial effort is being devoted to providing high-quality patient-centered care. Providing patient-centered care need not divert resources away from other quality improvement efforts. Improving the infrastructure supporting certain aspects of care may have broad effects because system changes can influence multiple outcomes. Thus, rather than detract from general quality improvement efforts, making changes that facilitate patient-centered care may lead to broader improvements. There is good reason to be optimistic that our health care system will increasingly be "patient centered."

Should health care providers be accountable for patients' care experiences?

Measures of patients' care experiences are increasingly used as quality measures in accountability initiatives. As the prominence and financial impact of patient experience measures have increased, so too have concerns about the relevance and fairness of including them as indicators of health care quality. Using evidence from the Consumer Assessment of Healthcare Providers and Systems (CAHPS®) surveys, the most widely used patient experience measures in the United States, we address seven common critiques of patient experience measures: (1) consumers do not have the expertise needed to evaluate care quality; (2) patient "satisfaction" is subjective and thus not valid or actionable; (3) increasing emphasis on improving patient experiences encourages health care providers and plans to fulfill patient desires, leading to care that is inappropriate, ineffective, and/or inefficient; (4) there is a trade-off between providing good patient experiences and providing high-quality clinical care; (5) patient scores cannot be fairly compared across health care providers or plans due to factors beyond providers' control; (6) response rates to patient experience surveys are low, or responses reflect only patients with extreme experiences; and (7) there are faster, cheaper, and more customized ways to survey patients than the standardized approaches mandated by federal accountability initiatives.

Clinician advice to quit smoking among seniors.

OBJECTIVE: Little smoking research in the past 20years includes persons 50 and older; herein we describe patterns of clinician cessation advice to US seniors, including variation by Medicare beneficiary characteristics. METHOD: In 2012-4, we analyzed 2010 Consumer Assessment of Healthcare Providers and Systems (CAHPS) survey data from Medicare beneficiaries over age 64 (n=346,674). We estimated smoking rates and the proportion of smokers whose clinicians encouraged cessation. RESULTS: 12% of male and 8% of female respondents aged 65 and older smoke. The rate decreases with age (14% of 65-69, 3% of 85+) and education (12-15% with no high school degree, 5-6% with BA+). Rates are highest among American Indian/Alaskan Native (16%), multiracial (14%), and African-American (13%) seniors, and in the Southeast (14%). Only 51% of smokers say they receive cessation advice "always" or "usually" at doctor visits, with advice more often given to the young, those in low-smoking regions, Asians, and women. For all results cited p<0.05. CONCLUSIONS: Smoking cessation advice to seniors is variable. Providers may focus on groups or areas in which smoking is less common or when they are most comfortable giving advice. More consistent interventions are needed, including cessation advice from clinicians.

The relationship between voice climate and patients' experience of timely care in primary care clinics.

BACKGROUND: Aspects of the patient care experience, despite being central to quality care, are often problematic. In particular, patients frequently report problems with timeliness of care. As yet, research offers little insight on setting characteristics that contribute to patients' experience of timely care. PURPOSE: The aims of this study were to assess the relationship between organizational climate and patients' reports of timely care in primary care clinics and to broadly examine the link between staff's work environment and patient care experiences. We test hypotheses about the relationship between voice climate--staff feeling safe to speak up about issues--and reported timeliness of care, consistency in reported voice climate across professions, and how climate differences for various professions relate to timely care. METHODOLOGY: We conducted a cross-sectional study of employees (n = 1,121) and patients (n = 8,164) affiliated with 37 clinics participating in a statewide reporting initiative. Employees were surveyed about clinics' voice climate, and patients were surveyed about the timeliness of care. Hypotheses were tested using analysis of variance and generalized estimating equations. FINDINGS: Clinical and administrative staff (e.g., nurses and office assistants) reported clinics' climates to be significantly less supportive of voice than did clinical leaders (e.g., physicians). The greater the difference in reported support for voice between professional groups, the less patients reported experiencing timely care in three respects: obtaining an appointment, seeing the doctor within 15 minutes of appointment time, and receiving test results. In clinics where staff reported climates supportive of voice, patients indicated receiving more timely care. Clinical leaders' reports of voice climate had no relationship to reported timeliness of care. PRACTICAL IMPLICATIONS: Our findings suggest the importance of clinics developing a strong climate for voice, particularly for clinical and administrative staff, to support better service quality for patients.

Development and evaluation of the CAHPS (Consumer Assessment of Healthcare Providers and Systems) survey for in-center hemodialysis patients.

BACKGROUND: The US Centers for Medicare & Medicaid Services assess patient experiences of care as part of the end-stage renal disease prospective payment system and Quality Incentive Program. This article describes the development and evaluation of the Consumer Assessment of Healthcare Providers and Systems (CAHPS) In-Center Hemodialysis Survey. STUDY DESIGN: We conducted formative research to generate survey questions and performed statistical analyses to evaluate the survey's measurement properties. SETTING & PARTICIPANTS: Formative research included focus groups, cognitive interviews, and field testing the survey with dialysis patients. MEASUREMENTS & OUTCOMES: We assessed internal consistency reliability (Cronbach alpha) and center-level reliability for 3 multi-item scales. We evaluated construct validity using correlations of the scales with global ratings of the kidney doctor, staff, and dialysis center. RESULTS: Response rate was 46% (1,454 completed surveys). Analyses support 3 multi-item scales: Nephrologists' Communication and Caring (7 items, alpha=0.89), Quality of Dialysis Center Care and Operations (22 items, alpha=0.93), and Providing Information to Patients (11 items, alpha=0.75). The communication scale was correlated the most strongly with the global rating of the "kidney doctor" (r=0.78). The Dialysis Center Care and Operations scale was correlated most strongly with global ratings of staff (r=0.75) and the center (r=0.69). Providing Information to Patients was correlated most strongly with the global rating of the staff (r=0.41). LIMITATIONS: A relatively small number of patients completed the survey in Spanish. CONCLUSIONS: This study provides support for the reliability and validity of the CAHPS In-Center Hemodialysis Survey for assessing patient experiences of care at dialysis facilities. The survey can be used to compare care provided at different facilities.

Are comparisons of patient experiences across hospitals fair? A study in Veterans Health Administration hospitals.

BACKGROUND: Surveys are increasingly used to assess patient experiences with health care. Comparisons of hospital scores based on patient experience surveys should be adjusted for patient characteristics that might affect survey results. Such characteristics are commonly drawn from patient surveys that collect little, if any, clinical information. Consequently some hospitals, especially those treating particularly complex patients, have been concerned that standard adjustment methods do not adequately reflect the challenges of treating their patients. OBJECTIVES: To compare scores for different types of hospitals after making adjustments using only survey-reported patient characteristics and using more complete clinical and hospital information. RESEARCH DESIGN: We used clinical and survey data from a national sample of 1858 veterans hospitalized for an initial acute myocardial infarction (AMI) in a Department of Veterans Affairs (VA) medical center during fiscal years 2003 and 2004. We used VA administrative data to characterize hospitals. The survey asked patients about their experiences with hospital care. The clinical data included 14 measures abstracted from medical records that are predictive of survival after an AMI. RESULTS: Comparisons of scores across hospitals adjusted only for patient-reported health status and sociodemographic characteristics were similar to those that also adjusted for patient clinical characteristics; the Spearman rank-order correlations between the 2 sets of adjusted scores were >0.97 across 9 dimensions of inpatient experience. CONCLUSIONS: This study did not support concerns that measures of patient care experiences are unfair because commonly used models do not adjust adequately for potentially confounding patient clinical characteristics.

Survey Protocols, Response Rates, and Representation of Underserved Patients: A Randomized Clinical Trial.

Importance: Surveys often underrepresent certain patients, such as underserved patients. Methods that improve their response rates (RRs) would help patient surveys better represent their experiences and assess equity and equity-targeted quality improvement efforts. Objective: To estimate the effect of adding an initial web mode to existing Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) survey protocols and extending the fielding period on RR and representativeness of underserved patient groups. Design, Setting, and Participants: This randomized clinical trial included 36 001 patients discharged from 46 US hospitals from May through December 2021. Data analysis was performed from May 2022 to September 2023. Exposures: Patients were randomized to 1 of 6 survey protocols: 3 standard HCAHPS protocols (mail only, phone only, mail-phone) plus 3 web-enhanced protocols (web-mail, web-phone, web-mail-phone). Main Outcomes and Measures: RR and number of respondents per 100 survey attempts (yield) were calculated and compared for each of the 6 survey protocols, overall, and by patient age, service line, sex, and race and ethnicity. Results: A total of 34 335 patients (median age range, 55-59 years; 59.3% female individuals and 40.7% male individuals) were eligible and included in the study. Of the respondents, 6.9% were Asian American or Native Hawaiian or Other Pacific Islander, 0.7% were American Indian or Alaska Native, 11.5% were Black, 17.4% were Hispanic, 61.0% were White, and 2.6% were multiracial. Of the 6 protocols, RRs were highest in web-mail-phone (36.5%), intermediate for the 3 two-mode survey protocols (mail-phone, web-mail, web-phone, 30.3%-31.1%), and lowest for the 2 single-mode protocols (mail only, phone only, 22.1%-24.3%). Web-mail-phone resulted in the highest yield for 3 racial and ethnic groups (Black, Hispanic, and White patients) and second highest for another (multiracial patients). Otherwise, the highest or second highest yield was almost always a 2-mode protocol. Mail only was the lowest-yield protocol for Black, Hispanic, and multiracial patients and phone-only was the lowest-yield protocol for White patients; these 2 protocols tied for lowest-yield for Asian American or Native Hawaiian or Other Pacific Islander patients. Gains from multimode approaches were often 2 to 3 times as large for Asian American or Native Hawaiian or Other Pacific Islander, Black, Hispanic, and multiracial patients as for White patients. Web-mail-phone had the highest RR for 6 of 8 age groups and 4 of 5 combinations of service line and sex. Conclusions and Relevance: In this randomized clinical trial, web-first multimode survey protocols significantly improved the RR and representativeness of patient surveys. The best-performing protocol based on RR and representativeness was web-mail-phone. Web-phone performed well for young and diverse patient populations, and web-mail for older and less diverse patient populations. The US Centers for Medicare & Medicaid Services will allow hospitals to use the web-mail, web-phone, and web-mail-phone protocols for HCAHPS administration beginning in 2025.

Changes in Patient Experiences of Hospital Care During the COVID-19 Pandemic.

Importance: It is important to assess how the COVID-19 pandemic was adversely associated with patients' care experiences. Objective: To describe differences in 2020 to 2021 patient experiences from what would have been expected from prepandemic (2018-2019) trends and assess correlates of changes across hospitals. Design, Setting, and Participants: This cohort study compared 2020 to 2021 data with 2018 to 2019 data from 3 900 887 HCAHPS respondents discharged from 3381 HCAHPS-participating US hospitals. The data were analyzed from 2022 to 2023. Main Outcomes and Measures: The primary outcome was an HCAHPS summary score (HCAHPS-SS), which averaged 10 HCAHPS measures. The primary analysis estimated whether HCAHPS scores from patients discharged from 2020 to 2021 differed from scores that would be expected based on quarterly and linear trends from 2018 to 2019 discharges. Secondary analyses stratified hospitals by prepandemic overall star ratings and staffing levels. Results: Of the 3 900 887 HCAHPS 2020 to 2021 respondents, 59% were age 65 years or older, and 35% (11%) were in the surgical (maternity) service lines. Compared with trends expected based on prepandemic (2018-2019) data, HCAHPS-SS was 1.2 percentage points (pp) lower for quarter (Q) 2/2020 discharges and -1.9 to -2.0 pp for Q3/2020 to Q1/2021, which then declined to -3.6 pp by Q4/2021. The most affected measures (Q4/2021) were staff responsiveness (-5.6 pp) and cleanliness (-4.9 pp); the least affected were discharge information (-1.6 pp) and quietness (-1.8 pp). Overall rating and hospital recommendation measures initially exhibited smaller-than-average decreases, but then fell as much as the more specific experience measures by Q2/2021. Quietness did not decline until Q2/2021. The HCAHPS-SS fell most for hospitals with the lowest prepandemic staffing levels; hospitals with bottom-quartile staffing showed the largest decrements, whereas top-quartile hospitals showed smaller decrements in most quarters. Hospitals with better overall prepandemic quality showed consistently smaller HCAHPS-SS drops, with effects for 5-star hospitals about 25% smaller than for 1-star and 2-star hospitals. Conclusions and Relevance: The results of this cohort study of HCAHPS-participating hospitals found that patient experience scores declined during 2020 to 2021. By Q4/2021, the HCAHPS-SS was 3.6 pp lower than would have been expected, a medium effect size. The most affected measures (staff responsiveness and cleanliness) showed large effect sizes, possibly reflecting high illness-associated hospital workforce absenteeism. Hospitals that were lower performing and less staffed prepandemic may have been less resilient to reduced staff availability and other pandemic-associated challenges. However, by Q4/2021, even prepandemic high-performing hospitals had similar declines.

Improving organizational climate for quality and quality of care: does membership in a collaborative help?

BACKGROUND: The lack of quality-oriented organizational climates is partly responsible for deficiencies in patient-centered care and poor quality more broadly. To improve their quality-oriented climates, several organizations have joined quality improvement collaboratives. The effectiveness of this approach is unknown. OBJECTIVE: To evaluate the impact of collaborative membership on organizational climate for quality and service quality. SUBJECTS: Twenty-one clinics, 4 of which participated in a collaborative sponsored by the Institute for Clinical Systems Improvement. RESEARCH DESIGN: Pre-post design. Preassessments occurred 2 months before the collaborative began in January 2009. Postassessments of service quality and climate occurred about 6 months and 1 year, respectively, after the collaborative ended in January 2010. We surveyed clinic employees (eg, physicians, nurses, receptionists, etc.) about the organizational climate and patients about service quality. MEASURES: Prioritization of quality care, high-quality staff relationships, and open communication as indicators of quality-oriented climate and timeliness of care, staff helpfulness, doctor-patient communication, rating of doctor, and willingness to recommend doctor's office as indicators of service quality. RESULTS: There was no significant effect of collaborative membership on quality-oriented climate and mixed effects on service quality. Doctors' ratings improved significantly more in intervention clinics than in control clinics, staff helpfulness improved less, and timeliness of care declined more. Ratings of doctor-patient communication and willingness to recommend doctor were not significantly different between intervention and comparison clinics. CONCLUSION: Membership in the collaborative provided no significant advantage for improving quality-oriented climate and had equivocal effects on service quality.