Survey Protocols, Response Rates, and Representation of Underserved Patients: A Randomized Clinical Trial.

Importance: Surveys often underrepresent certain patients, such as underserved patients. Methods that improve their response rates (RRs) would help patient surveys better represent their experiences and assess equity and equity-targeted quality improvement efforts. Objective: To estimate the effect of adding an initial web mode to existing Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) survey protocols and extending the fielding period on RR and representativeness of underserved patient groups. Design, Setting, and Participants: This randomized clinical trial included 36 001 patients discharged from 46 US hospitals from May through December 2021. Data analysis was performed from May 2022 to September 2023. Exposures: Patients were randomized to 1 of 6 survey protocols: 3 standard HCAHPS protocols (mail only, phone only, mail-phone) plus 3 web-enhanced protocols (web-mail, web-phone, web-mail-phone). Main Outcomes and Measures: RR and number of respondents per 100 survey attempts (yield) were calculated and compared for each of the 6 survey protocols, overall, and by patient age, service line, sex, and race and ethnicity. Results: A total of 34 335 patients (median age range, 55-59 years; 59.3% female individuals and 40.7% male individuals) were eligible and included in the study. Of the respondents, 6.9% were Asian American or Native Hawaiian or Other Pacific Islander, 0.7% were American Indian or Alaska Native, 11.5% were Black, 17.4% were Hispanic, 61.0% were White, and 2.6% were multiracial. Of the 6 protocols, RRs were highest in web-mail-phone (36.5%), intermediate for the 3 two-mode survey protocols (mail-phone, web-mail, web-phone, 30.3%-31.1%), and lowest for the 2 single-mode protocols (mail only, phone only, 22.1%-24.3%). Web-mail-phone resulted in the highest yield for 3 racial and ethnic groups (Black, Hispanic, and White patients) and second highest for another (multiracial patients). Otherwise, the highest or second highest yield was almost always a 2-mode protocol. Mail only was the lowest-yield protocol for Black, Hispanic, and multiracial patients and phone-only was the lowest-yield protocol for White patients; these 2 protocols tied for lowest-yield for Asian American or Native Hawaiian or Other Pacific Islander patients. Gains from multimode approaches were often 2 to 3 times as large for Asian American or Native Hawaiian or Other Pacific Islander, Black, Hispanic, and multiracial patients as for White patients. Web-mail-phone had the highest RR for 6 of 8 age groups and 4 of 5 combinations of service line and sex. Conclusions and Relevance: In this randomized clinical trial, web-first multimode survey protocols significantly improved the RR and representativeness of patient surveys. The best-performing protocol based on RR and representativeness was web-mail-phone. Web-phone performed well for young and diverse patient populations, and web-mail for older and less diverse patient populations. The US Centers for Medicare & Medicaid Services will allow hospitals to use the web-mail, web-phone, and web-mail-phone protocols for HCAHPS administration beginning in 2025.

Changes in Patient Experiences of Hospital Care During the COVID-19 Pandemic.

Importance: It is important to assess how the COVID-19 pandemic was adversely associated with patients' care experiences. Objective: To describe differences in 2020 to 2021 patient experiences from what would have been expected from prepandemic (2018-2019) trends and assess correlates of changes across hospitals. Design, Setting, and Participants: This cohort study compared 2020 to 2021 data with 2018 to 2019 data from 3 900 887 HCAHPS respondents discharged from 3381 HCAHPS-participating US hospitals. The data were analyzed from 2022 to 2023. Main Outcomes and Measures: The primary outcome was an HCAHPS summary score (HCAHPS-SS), which averaged 10 HCAHPS measures. The primary analysis estimated whether HCAHPS scores from patients discharged from 2020 to 2021 differed from scores that would be expected based on quarterly and linear trends from 2018 to 2019 discharges. Secondary analyses stratified hospitals by prepandemic overall star ratings and staffing levels. Results: Of the 3 900 887 HCAHPS 2020 to 2021 respondents, 59% were age 65 years or older, and 35% (11%) were in the surgical (maternity) service lines. Compared with trends expected based on prepandemic (2018-2019) data, HCAHPS-SS was 1.2 percentage points (pp) lower for quarter (Q) 2/2020 discharges and -1.9 to -2.0 pp for Q3/2020 to Q1/2021, which then declined to -3.6 pp by Q4/2021. The most affected measures (Q4/2021) were staff responsiveness (-5.6 pp) and cleanliness (-4.9 pp); the least affected were discharge information (-1.6 pp) and quietness (-1.8 pp). Overall rating and hospital recommendation measures initially exhibited smaller-than-average decreases, but then fell as much as the more specific experience measures by Q2/2021. Quietness did not decline until Q2/2021. The HCAHPS-SS fell most for hospitals with the lowest prepandemic staffing levels; hospitals with bottom-quartile staffing showed the largest decrements, whereas top-quartile hospitals showed smaller decrements in most quarters. Hospitals with better overall prepandemic quality showed consistently smaller HCAHPS-SS drops, with effects for 5-star hospitals about 25% smaller than for 1-star and 2-star hospitals. Conclusions and Relevance: The results of this cohort study of HCAHPS-participating hospitals found that patient experience scores declined during 2020 to 2021. By Q4/2021, the HCAHPS-SS was 3.6 pp lower than would have been expected, a medium effect size. The most affected measures (staff responsiveness and cleanliness) showed large effect sizes, possibly reflecting high illness-associated hospital workforce absenteeism. Hospitals that were lower performing and less staffed prepandemic may have been less resilient to reduced staff availability and other pandemic-associated challenges. However, by Q4/2021, even prepandemic high-performing hospitals had similar declines.

A Systematic Review of Strategies to Enhance Response Rates and Representativeness of Patient Experience Surveys.

BACKGROUND: Data from surveys of patient care experiences are a cornerstone of public reporting and pay-for-performance initiatives. Recently, increasing concerns have been raised about survey response rates and how to promote equity by ensuring that responses represent the perspectives of all patients. OBJECTIVE: Review evidence on survey administration strategies to improve response rates and representativeness of patient surveys. RESEARCH DESIGN: Systematic review adhering to the Preferred Reporting Items for Systematic reviews and Meta-Analyses guidelines. STUDY SELECTION: Forty peer-reviewed randomized experiments of administration protocols for patient experience surveys. RESULTS: Mail administration with telephone follow-up provides a median response rate benefit of 13% compared with mail-only or telephone-only. While surveys administered only by web typically result in lower response rates than those administered by mail or telephone (median difference in response rate: -21%, range: -44%, 0%), the limited evidence for a sequential web-mail-telephone mode suggests a potential response rate benefit over sequential mail-telephone (median: 4%, range: 2%, 5%). Telephone-only and sequential mixed modes including telephone may yield better representation across patient subgroups by age, insurance type, and race/ethnicity. Monetary incentives are associated with large increases in response rates (median increase: 12%, range: 7%, 20%). CONCLUSIONS: Sequential mixed-mode administration yields higher patient survey response rates than a single mode. Including telephone in sequential mixed-mode administration improves response among those with historically lower response rates; including web in mixed-mode administration may increase response at lower cost. Other promising strategies to improve response rates include in-person survey administration during hospital discharge, incentives, minimizing survey language complexity, and prenotification before survey administration.

The hospital care experiences of older patients compared to younger patients.

BACKGROUND: Hospitals may provide less positive patient experiences for older than younger patients. METHODS: We used 2019 HCAHPS data from 4358 hospitals to compare patient-mix adjusted HCAHPS Survey scores for 19 experience of care items for patients ages 75+ versus 55-74 years and tested for interactions of age group with patient and hospital characteristics. We contrasted the age patterns observed for inpatient experiences with those among respondents to the 2019 Medicare CAHPS (MCAHPS) Survey of overall experience. RESULTS: Patients 75+ years (31% of all HCAHPS respondents) reported less-positive experiences than those 55-74 (46% of respondents) for 18 of 19 substantive HCAHPS items (mean difference -3.3% points). Age differences in HCAHPS top-box scores were large (>5 points) for 1 of 3 Nurse Communication items, 1 of 3 Doctor Communication, 2 of 2 Communication about Medication items, 1 of 2 Discharge Information items, and 2 of 3 Care Transition items. In contrast, for MCAHPS, those 75+ reported similar experiences to younger adults. The magnitude of age differences varied considerably across hospitals; some hospitals had very large age disparities for older patients (age 75+ vs. ages 55-74), while others had none. These age differences were generally smaller for patients in government and non-profit than in for-profit hospitals, and in the Pacific region than in other parts of the United States. This variation in age disparities across hospitals may help to identify best practices. CONCLUSIONS: Patients ages 75+ reported less-positive experiences than patients ages 55-74, especially for measures of communication. These differences may be specific to inpatient care. Further study should investigate the effectiveness of hospital staffs' communication with older patients. Hospital protocols designed for younger patients may need to be adjusted to meet the needs of older patients. There may also be opportunities to learn from outpatient interactions with older patients.

How responding in Spanish affects CAHPS results.

BACKGROUND: The most widely used surveys for assessing patient health care experiences in the U.S. are the Consumer Assessment of Healthcare Providers and Systems (CAHPS) surveys. Studies examining the associations of language and ethnicity with responses to CAHPS surveys have yielded inconsistent findings. More research is needed to assess the effect of responding to CAHPS surveys in Spanish. METHODS: Subjects were patients who had received care at a study community health center in Connecticut within 6 or 12 months of being sent a CAHPS survey that asks about care experiences. The survey included four multi-item measures of care plus an overall rating of the provider. Sampled patients were mailed dual language (English and Spanish) cover letters and questionnaires. Those who did not respond after follow-up mailings were contacted by bilingual interviewers to complete the survey by telephone. We tested three hypotheses for any observed differences by ethnicity and language: 1. Spanish speakers are more likely than others to choose extreme response options. 2. The semantic meaning of the Spanish translation is not the same as the English version of the questions, resulting in Spanish speakers giving different answers because of meaning differences. 3. Spanish speakers have different expectations regarding their health care than those who answer in English. Analyses compared the answers on the survey measures for three groups: non-Hispanics answering in English, Hispanics answering in English, and Hispanics answering in Spanish. RESULTS: The overall response rate was 45%. After adjusting for differences in demographic characteristics and self-rated health, those answering in Spanish gave significantly more positive reports than the other two groups on three of the five measures, and higher than the non-Hispanic respondents on a fourth. CONCLUSIONS: Those answering in Spanish gave more positive reports of their medical experiences than Hispanics and non-Hispanics answering in English. Whether these results reflect different response tendencies, different standards for care, or better care experiences is a key issue in whether CAHPS responses in Spanish need adjustment to make them comparable to responses in English.

Summary of the 2020 AHRQ research meeting on 'advancing methods of implementing and evaluating patient experience improvement using consumer assessment of healthcare providers and systems (CAHPS®) surveys'.

BACKGROUND: The Agency for Healthcare Research and Quality held a research meeting on using Consumer Assessment of Healthcare Providers and Systems (CAHPS®) data for quality improvement (QI) and evaluating such efforts. TOPICS COVERED.: Meeting addressed: 1)What has been learned about organizational factors/environment needed to improve patient experience? 2)How have organizations used data to improve patient experience? 3)What can evaluations using CAHPS data teach us about implementing successful programs to improve patient experience? KEY THEMES: Providers and stakeholders need to be engaged early and often, standardize QI processes, complement CAHPS data with other data, and compile dashboards of CAHPS scores to identify and track improvement. Rigorous study designs are valuable, but much can be learned and accomplished through practical organization-level studies.

Adjusting for Patient Characteristics to Compare Quality of Care Provided by Serious Illness Programs.

Background: To compare serious illness programs (SIPs) using recently developed patient experience measures, adjustment must be made for patient characteristics not under control of the programs. Objectives: To develop a case-mix adjustment model to enable fair comparison of patient experience between SIPs by investigating the roles of patient characteristics, proxy response, and mode of survey administration (mail-only vs. mail with telephone follow-up) in survey responses. Methods: Using survey data from 2263 patients from 32 home-based SIPs across the United States, we fit regression models to assess the association between patient-level variables and scores for seven quality measures (Communication, Care Coordination, Help for Symptoms, Planning for Care, Support for Family and Friends, and two global assessments of care). Characteristics that are not consequences of the care the program delivered were considered as adjustors. Results: Final recommended case-mix adjustors are age, education, primary diagnosis, self-reported functional status, self-rated physical health, self-rated mental health, proxy respondent use, and response percentile (a measure of how soon a person responded compared with others in the same program and mode). Age, primary diagnosis, self-rated mental health, and proxy respondent use had the most impact on program-level scores. We also recommend adjusting for mode of survey administration. We find that up to 12 percent of pairs of programs would have their rankings reversed by adjustment. Conclusions: To ensure fair comparison of programs, scores should be case-mix adjusted for variables that influence patients' reports about care quality, but are not under the control of the program administering care.

Reliable and Valid Survey-Based Measures to Assess Quality of Care in Home-Based Serious Illness Programs.

Background: There is a pressing need for standardized measures to assess the quality of home-based serious illness care. Currently, there are no validated quality measures that are specific to home-based serious illness programs (SIPs) and the unique needs of their patients. Objective: To develop and evaluate standardized survey-based measures of serious illness care experiences for assessing and comparing quality of home-based serious illness care programs. Methods: From October 2019 through January 2020, we administered a survey to patients who received care from 32 home-based SIPs across the United States. Using the 2263 survey responses, we assessed item performance and constructed composite measures via factor analysis, evaluated item-scale correlations, estimated reliability, and examined validity by regressing overall ratings and willingness to recommend care on each composite. Results: The overall survey response rate was 36%. Confirmatory factor analyses supported five composite quality measures: Communication, Care Coordination, Help for Symptoms, Planning for Care, and Support for Family and Friends. Cronbach's alpha estimates for the composite measures ranged from 0.69 to 0.85, indicating adequate internal consistency in assessing their underlying constructs. Interprogram reliability ranged from 0.67 to 0.80 at 100 completed surveys per measure, meeting common standards for distinguishing between programs' performance. Together, the composites explained 45% of the variance in patients' overall care ratings. Communication, Care Coordination, and Planning for Care were the strongest predictors of overall ratings. Conclusion: Our analyses provide evidence of the feasibility, reliability, and validity of proposed survey-based measures to assess the quality of home-based serious illness care from the perspective of patients and their families.

Comparing Web and Mail Protocols for Administering Hospital Consumer Assessment of Healthcare Providers and Systems Surveys.

OBJECTIVE: The objective of this study was to compare results of using web-based and mail (postal) Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) data collection protocols. RESEARCH DESIGN: Patients who had been hospitalized in a New England Hospital were surveyed about their hospital experience. Patients who provided email addresses were randomized to 1 of 3 data collection protocols: web-alone, web with postal mail follow-up, and postal mail only. Those who did not provide email addresses were surveyed using postal mail only. Analyses compared response rates, respondent characteristics, and patient-reported experiences. SUBJECTS: For an 8-week period, patients were discharged from the study hospital to home. MEASURES: Measures included response rates, characteristics of respondents, 6 composite measures of their patient experiences, and 2 ratings of the hospital. RESULTS: Response rates were significantly lower for the web-only protocol than the mail or combined protocols, and those who had not provided email addresses had lower response rates. Those over 65 were more likely than others to respond to all protocols, especially for the mail-only protocols. Respondents without email addresses were older, less educated, and reported worse health than those who provided email addresses. After adjusting for respondent differences, those in the combined protocol differed significantly from the mail (postal) only respondents on 2 measures of patient experience; those in the web-only protocol differed on one. Those not providing an email address differed from those who did on one measure. CONCLUSION: If web-based protocols are used for HCAHPS surveys, adjustments for a mode of data collection are needed to make results comparable.

The development and evaluation of an HIV implementation science network in New England: lessons learned.

BACKGROUND: Describe and evaluate an implementation science network focused on HIV prevention and treatment in New England. METHODS: In 2014, we established a partnership among university researchers and community stakeholders to stimulate and support HIV-related implementation research. We solicited information from Network members through surveys, interviews at Network events, and dialog with participants. In 2017, we conducted a sociocentric network assessment of collaborations on research projects, grants, manuscripts, and consultations. RESULTS: We identified 988 connections made through the Network that resulted in 185 manuscripts published and 15 grants funded. Our experience indicated that eight factors were instrumental in building and sustaining the Network: (1) acknowledging different perspectives, (2) balancing content and expertise, (3) encouraging consistent engagement, (4) providing seed funding, (5) membership flexibility, (6) maintenance of Network interactions, (7) supporting local HIV prevention and treatment efforts, and (8) maintaining productive relationships with health departments and community-based organizations. CONCLUSIONS: Developing and maintaining a regional network on implementation science for HIV prevention and treatment is feasible and can facilitate new and productive partnerships among researchers and community organizations and members.

Is Patient-Physician Gender Concordance Related to the Quality of Patient Care Experiences?

BACKGROUND: There is great interest in identifying factors that are related to positive patient experiences such as physician communication style. Documented gender-specific physician communication and patient behavior differences raise the question of whether gender concordant relationships (i.e., both the provider and patient share the same gender) might affect patient experiences. OBJECTIVE: Assess whether patient experiences are more positive in gender concordant primary care relationships. DESIGN: Statewide telephone surveys. Linear mixed regression models to estimate the association of CAHPS scores with patient gender and gender concordance. SUBJECTS: Two probability samples of primary care Medicaid patients in Connecticut in 2017 (5/17-7/17) and 2019 (7/19-10/19). MAIN MEASURES: Clinician and Group Consumer Assessment of Healthcare Providers and Systems (CG-CAHPS) survey augmented with questions about aspects of care most salient to PCMH-designated organizations and two questions to assess access to mental health services. KEY RESULTS: There were no significant effects of gender concordance and differences in experiences by patient gender were modest. CONCLUSIONS: This study did not support the suggestion that patient and physician gender and gender concordance have an important effect on patient experiences.

Associations of Mail Survey Length and Layout With Response Rates.

We assess the association between survey layout and response rates (RRs) in the 2017 Medicare Advantage Consumer Assessment of Healthcare Providers and Systems mail survey. Among 438 Medicare Advantage plans surveyed by six vendors, there was latitude in survey layout, and plans could add up to 12 supplemental items. Regression models predicted survey response from survey characteristics (page count, number of supplemental items, and survey attractiveness), and beneficiary sociodemographics. Beneficiary-age-by-survey-characteristic interactions assessed whether survey characteristics were more strongly related to RRs among older beneficiaries. We found that surveys with more supplemental items and less attractive layouts had lower adjusted odds of response. RRs were more sensitive to format among older beneficiaries. The difference in adjusted RRs for the most favorable versus the least favorable survey design was 14.5%. For a 65-year-old, this difference was 13.6%; for an 80-year-old, it was 21.0%. These findings suggest that even within a relatively standardized survey, formatting can substantially influence RRs.

Effects of Leader Tactics on the Creativity, Implementation, and Evolution of Ideas to Improve Healthcare Delivery.

BACKGROUND: Slow progress in quality improvement (QI) has prompted calls to identify new QI ideas. Leaders guiding these efforts are advised to use evidence-based tactics, or specific approaches to address a goal, to promote clinician and staff engagement in the generation and implementation of QI ideas, but little evidence about effective tactics exists. OBJECTIVE: Examine the association between leader tactics and the creativity, implementation outcome, and evolution of QI ideas from clinicians and staff. DESIGN: Prospective panel analysis of 220 ideas generated by 12 leaders and teams (N = 72 members) from federally qualified community health practices in one center over 18 months. Measures were extracted from meeting minutes (note-taking by a member during meetings) and expert panel review. Multi-level models were used. MEASURES: Leader tactics, idea creativity, implementation outcome, evolution pathways, center, and idea-submitter characteristics. RESULTS: Leaders used one of four approaches: no tactic, meeting ground rules, team brainstorming, or reflection on team process. Implemented ideas evolved in three pathways: Plug and Play, Slow Burn, and Iterate and Generate. Compared with no leader tactic, meeting ground rules resulted in ideas not significantly different in creativity, implementation outcome, or evolution pathway. Brainstorming was associated with greater idea creativity, idea implementation, and ideas following a Plug and Play path (low member engagement and implementation over 2 months or less). Reflection on team process was associated with idea implementation (versus not), and ideas following an Iterate and Generate path (high member engagement and implementation over 3 months or more). CONCLUSIONS: Two tactics, brainstorming and reflection, are helpful depending on goals. Brainstorming may aide leaders seeking disruptive change via more creative, rapidly implemented ideas. Reflection on team process may aide leaders seeking high-engagement ideas that may be implemented slowly. Both tactics may help leaders cultivate dynamics that increase implementation of ideas that improve healthcare.

The How Matters: How Primary Care Provider Communication With Team Relates to Patients' Disease Management.

BACKGROUND: Investigating primary care provider (PCP)-team communication can provide insight into how colleagues work together to become high-functioning teams more able to address an increasingly complex set of tasks associated with chronic disease management. OBJECTIVE: To assess how PCP communication with their care team relates to patients' health. RESEARCH DESIGN: Longitudinal study of how 3 aspects of PCP-care team communication-participation, time spent listening, and uninterrupted speaking length-relate to disease management of patients with hypertension or diabetes, and the effect of these team communication behaviors on PCP-patient communication as a pathway by which this relationship might exist. We used multilevel regression models. SUBJECTS: Twenty-seven PCPs and 98 team members, and 18,067 patients with hypertension and 8354 patients with diabetes affiliated with a federally qualified health center with 12 practice sites. MEASURES: Primary data on communication collected using sociometric sensors worn by PCPs and team members, patient-PCP communication data collected with surveys, and patient health, PCP and patient characteristics extracted from electronic records. RESULTS: PCPs participated in 75% of care team conversations, spent 56% of conversation time listening, and had an average uninterrupted speaking length of 2.42 seconds. PCP participation, listening, and length of uninterrupted speaking time were associated with significantly higher odds that their patients had controlled hypertension and diabetes and improvements in disease control over time. PCP-patient communication mediates this relationship. CONCLUSIONS: PCP-team communication is associated with patient health management. How team members speak with one another may be as important as the content of their communication.

A quasi-experiment assessing the six-months effects of a nurse care coordination program on patient care experiences and clinician teamwork in community health centers.

BACKGROUND: Recognition that coordination among healthcare providers is associated with better quality of care and lower costs has increased interest in interventions designed to improve care coordination. One intervention is to add care coordination to nurses' role in a formal way. Little is known about effects of this approach, which tends to be pursued by small organizations and those in lower-resource settings. We assessed effects of this approach on care experiences of high-risk patients (those most in need of care coordination) and clinician teamwork during the first 6 months of use. METHODS: We conducted a quasi-experimental study using a clustered, controlled pre-post design. Changes in staff and patient experiences at six community health center practice locations that introduced the added-role approach for high-risk patients were compared to changes in six locations without the program in the same health system. In the pre-period (6 months before intervention training) and post-period (about 6 months after intervention launch, following 3 months of training), we surveyed clinical staff (N = 171) and program-qualifying patients (3007 pre-period; 2101 post-period, including 113 who were enrolled during the program's first 6 months). Difference-in-differences models examined study outcomes: patient reports about care experiences and clinician-reported teamwork. We assessed frequency of patient office visits to validate access and implementation, and contextual factors (training, resources, and compatibility with other work) that might explain results. RESULTS: Patient care experiences across all high-risk patients did not improve significantly (p > 0.05). They improved somewhat for program enrollees, 5% above baseline reports (p = 0.07). Staff-perceived teamwork did not change significantly (p = 0.12). Office visits increased significantly for enrolled patients (p < 0.001), affirming program implementation (greater accessing of care). Contextual factors were not reported as problematic, except that 41% of nurses reported incompatibility between care coordination and other job demands. Over 75% of nurses reported adequate training and resources. CONCLUSIONS: There were some positive effects of adding care coordination to nurses' role within 6 months of implementation, suggesting value in this improvement strategy. Addressing compatibility between coordination and other job demands is important when implementing this approach to coordination.

The effects of survey version on patient experience scores and plan rankings.

OBJECTIVE: To assess the effect of changing survey questions on plan-level patient experience measures and ratings. DATA SOURCE: 2015 Medicare Advantage CAHPS Survey respondents. STUDY DESIGN: Ninety three randomly selected beneficiaries in each of 40 MA plans received a revised (5.0) CAHPS survey; 38 832 beneficiaries received version 4.0. Linear mixed-effect regression predicted CAHPS measures from fixed effects for survey version and beneficiary characteristics and random effects for plan and plan-by-version random slope. PRINCIPAL FINDINGS: Response rates were 42 percent for both versions. Removal of "try to" from screeners increased the percentage of respondents eligible for follow-up questions. Version 5.0 caused a small increase (1-3 points on a 0-100 scale, P < 0.05) in the mean of three altered measures and a moderate increase (>3 points) in one. There was a small statistically significant increase in two unaltered measures. These changes were uniform across plans, so there would be no expected change compared to results using the legacy survey in the score distributions other than uniform mean shifts, and no expected effect on summary measures. CONCLUSIONS: These analyses illustrate how to assess the impact of seemingly minor survey modifications for other national surveys considering changes and highlight the importance of screeners in instrument design.

The effect of different sampling and recall periods in the CAHPS Clinician & Group (CG-CAHPS) survey.

OBJECTIVE: To examine the effect of changing the sampling and reference periods for the CAHPS® Clinician & Group Survey from 12 to 6 months. DATA SOURCES/STUDY SETTING: Adult patients with a visit in the last 12 months to New England community health centers. STUDY DESIGN: We randomly assigned patients to receive a survey with either a 12- or 6-month recall period. DATA COLLECTION/EXTRACTION METHODS: Questionnaires were mailed to patients, with a second questionnaire mailed to nonrespondents, followed by six attempts to complete a telephone interview. PRINCIPAL FINDINGS: If the sampling criterion was a visit in the last 6 months, 9 percent of those with a visit in the last 12 months would not have been surveyed. A total of 1837 patients completed 6-month surveys (44.9 percent response rate); 588 completed 12-month surveys (46.0 percent response rate). Shortening the reference from 12 to 6 months reduced the proportion of respondents reporting a blood test, X-ray, or other tests. Adjusting for respondent characteristics, the most positive response was selected more often on the 6-month survey for 12 out of 13 questions, and three of these differences were statistically significant (P < 0.05). CONCLUSIONS: Surveys using a 6-month recall period may yield slightly higher scores than surveys with a 12-month recall period.