Validation of the Trust in the Surgical Decision Scale.

OBJECTIVE: To develop and validate a short measure of trust in the surgical decision making process. SUMMARY OF BACKGROUND DATA: Having a reliable and valid measure of trust is important to assess the quality of the patient-surgeon relationship when decisions about surgical procedures are made. METHODS: A previously published 10-item trust scale was qualitatively tested with patients, and a revised set of 14 items was tested using a web-based survey of 300 people who had hip, knee or back surgery in the past 2 years. The 14 items were evaluated using patterns of correlations and relevance to medical decision making to create a 5-item version. A 5-item subset was compared to the 14-item version to assess reliability and validity of patient's trust in the surgical decision making process. RESULTS: Of the 300 participants, 32% had hip surgery, 33% had knee surgery, and 34% back surgery. Mean age was 53 years, 45% female, 80% White, and 36% had a high school degree or less. The item intercorrelations for the 14 items were 0.43-0.72 and 0.58-0.71 for the 5 items. Correlation between the versions was 0.96 (P < 0.01). The 14- and 5-item versions were positively correlated with participants' shared decision making process scores (0.42 and 0.41, both P = 0.01), internal consistency reliability scores were 0.95 and 0.89, respectively, and were negatively correlated with their Decision Regret scores (-0.51 and -0.48, both P = 0.01). CONCLUSION: The 5-item Trust in the Surgical Decision Scale has strong evidence of validity and reliability for patients who underwent common orthopedic procedures.

How responding in Spanish affects CAHPS results.

BACKGROUND: The most widely used surveys for assessing patient health care experiences in the U.S. are the Consumer Assessment of Healthcare Providers and Systems (CAHPS) surveys. Studies examining the associations of language and ethnicity with responses to CAHPS surveys have yielded inconsistent findings. More research is needed to assess the effect of responding to CAHPS surveys in Spanish. METHODS: Subjects were patients who had received care at a study community health center in Connecticut within 6 or 12 months of being sent a CAHPS survey that asks about care experiences. The survey included four multi-item measures of care plus an overall rating of the provider. Sampled patients were mailed dual language (English and Spanish) cover letters and questionnaires. Those who did not respond after follow-up mailings were contacted by bilingual interviewers to complete the survey by telephone. We tested three hypotheses for any observed differences by ethnicity and language: 1. Spanish speakers are more likely than others to choose extreme response options. 2. The semantic meaning of the Spanish translation is not the same as the English version of the questions, resulting in Spanish speakers giving different answers because of meaning differences. 3. Spanish speakers have different expectations regarding their health care than those who answer in English. Analyses compared the answers on the survey measures for three groups: non-Hispanics answering in English, Hispanics answering in English, and Hispanics answering in Spanish. RESULTS: The overall response rate was 45%. After adjusting for differences in demographic characteristics and self-rated health, those answering in Spanish gave significantly more positive reports than the other two groups on three of the five measures, and higher than the non-Hispanic respondents on a fourth. CONCLUSIONS: Those answering in Spanish gave more positive reports of their medical experiences than Hispanics and non-Hispanics answering in English. Whether these results reflect different response tendencies, different standards for care, or better care experiences is a key issue in whether CAHPS responses in Spanish need adjustment to make them comparable to responses in English.

Comparing Web and Mail Protocols for Administering Hospital Consumer Assessment of Healthcare Providers and Systems Surveys.

OBJECTIVE: The objective of this study was to compare results of using web-based and mail (postal) Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) data collection protocols. RESEARCH DESIGN: Patients who had been hospitalized in a New England Hospital were surveyed about their hospital experience. Patients who provided email addresses were randomized to 1 of 3 data collection protocols: web-alone, web with postal mail follow-up, and postal mail only. Those who did not provide email addresses were surveyed using postal mail only. Analyses compared response rates, respondent characteristics, and patient-reported experiences. SUBJECTS: For an 8-week period, patients were discharged from the study hospital to home. MEASURES: Measures included response rates, characteristics of respondents, 6 composite measures of their patient experiences, and 2 ratings of the hospital. RESULTS: Response rates were significantly lower for the web-only protocol than the mail or combined protocols, and those who had not provided email addresses had lower response rates. Those over 65 were more likely than others to respond to all protocols, especially for the mail-only protocols. Respondents without email addresses were older, less educated, and reported worse health than those who provided email addresses. After adjusting for respondent differences, those in the combined protocol differed significantly from the mail (postal) only respondents on 2 measures of patient experience; those in the web-only protocol differed on one. Those not providing an email address differed from those who did on one measure. CONCLUSION: If web-based protocols are used for HCAHPS surveys, adjustments for a mode of data collection are needed to make results comparable.

Validation of the Impact Index: can we measure disease effects on quality of life in patients with hip and knee osteoarthritis?

PURPOSE: To validate the Impact Index, a short, publicly available scale that measures the extent to which a respondent's health problem adversely impacts their quality of life. METHODS: Secondary analysis of patients with hip or knee osteoarthritis surveyed after visiting a surgeon at baseline (N = 322) and about 6 months after the visit (N = 283). Patients responded to the Impact Index and previously validated questionnaires about overall health, pain, and function. The Impact Index includes four questions that ask how much the respondent is bothered, worried, limited, or in pain due to their health condition over the past 30 days. Total scores range from 0 to 12; higher scores indicate more deleterious impact. RESULTS: Patients were mostly female (55%), majority white (95%), had an average age of 65 (SD = 9), and most had surgery (64%). The baseline Impact Index score was 9.48 (SD = 2.63); at follow up 4.75 (SD = 3.54). Impact Index was related to overall health at baseline (r = - 0.49). For knee patients at baseline, Impact Index was negatively related to their knee symptoms (r = - 0.49) and knee pain (r = - 0.67). For hip patients at baseline, Impact Index was negatively related to the Harris Hip score (r = - 0.62). Scale directions varied; however, the signs of all correlations were as hypothesized. The Impact Index was predictive of surgical choice (p < .001, OR = 1.45), however, overall health (p = .88) and comorbidity (p = .24) measures were not. Reliability was acceptable (α = 0.85). Responsiveness statistics suggested overall health, pain, function, and Impact Index measures reflected improvement patients experienced from surgery. The Impact Index had the largest effect sizes (> - 3.4) and Guyatt Responsiveness Statistics (> - 2.3). CONCLUSIONS: The Impact Index demonstrated strong evidence of validity, reliability, and responsiveness in hip or knee osteoarthritis patients.

The American Urological Association Symptom Index for Benign Prostatic Hyperplasia.

A symptom index for benign prostatic hyperplasia (BPH) was developed and validated by a multidisciplinary measurement committee of the American Urological Association (AUA). Validation studies were conducted involving a total of 210 BPH patients and 108 control subjects. The final AUA symptom index includes 7 questions covering frequency, nocturia, weak urinary stream, hesitancy, intermittence, incomplete emptying and urgency. On revalidation, the index was internally consistent (Cronbach's α = 0.86) and the score generated had excellent test-retest reliability (r = 0.92). Scores were highly correlated with subjects' global ratings of the magnitude of their urinary problem (r = 0.65 to 0.72) and powerfully discriminated between BPH and control subjects (receiver operating characteristic area 0.85). Finally, the index was sensitive to change, with preoperative scores decreasing from a mean of 17.6 to 7.1 by 4 weeks after prostatectomy (p <0.001). The AUA symptom index is clinically sensible, reliable, valid and responsive. It is practical for use in practice and for inclusion in research protocols.

Validation of a New Three-Item Self-Report Measure for Medication Adherence.

Few self-report measures of medication adherence have been rigorously developed and validated against electronic drug monitoring (EDM). Assess the validity of the 3-item self-report scale by comparing it with a contemporaneous EDM measure. We conducted an observational study in which adherence assessments were done monthly for up to 4 months for 81 patients with HIV who were taking antiretroviral medications. We report results for both HIV antiretroviral medications, and also for other, non-HIV-related medications. Raw and calibrated self-report adherence measures, electronic drug monitoring adherence measures, and sociodemographic variables. The mean age of patients was 46 years, 37 % were female, 49 % had some education beyond high school, 22 % were Black, and 22 % were Hispanic. Cronbach's alphas for the 3-item scale for HIV and non-HIV medications were 0.83 and 0.87, respectively. The mean differences (raw/uncalibrated self-report scale minus EDM) for HIV and non-HIV medications were 7.5 and 5.2 points on a 100-point scale (p < 0.05 for both). Pearson correlation coefficients between the calibrated 3-item scale and the EDM for HIV and non-HIV medications were 0.47 and 0.59, respectively. The c-statistics for the ROC curves for the calibrated scale, using cut-offs of 0.8 and 0.9 for the EDM gold standard measure to define non-adherence, were between 0.74 and 0.76 for HIV and non-HIV medications. This 3-item adherence self-report scale showed good psychometric characteristics and good construct validity when compared with an EDM standard, for both HIV and non-HIV medications. In clinical care it can be a useful first-stage screener for non-adherence. In clinical research and quality improvement settings it can be a useful tool when more complex and expensive methods such as EDM or pharmacy claims are impractical or unavailable.

Cognitive and field testing of a new set of medication adherence self-report items for HIV care.

We conducted four rounds of cognitive testing of self-report items that included 66 sociodemographically diverse participants, then field tested the three best items from the cognitive testing in a clinic waiting room (N = 351) and in an online social networking site for men who have sex with men (N = 6,485). As part of the online survey we conducted a randomized assessment of two versions of the adherence questionnaire-one which asked about adherence to a specific antiretroviral medication, and a second which asked about adherence to their "HIV medicines" as a group. Participants were better able to respond using adjectival and adverbial scales than visual analogue or percent items. The internal consistency reliability of the three item adherence scale was 0.89. Mean scores for the two different versions of the online survey were similar (91.0 vs. 90.2, p < 0.05), suggesting that it is not necessary, in general, to ask about individual medications in an antiretroviral therapy regimen when attempting to describe overall adherence.

The role of perceived benefits and costs in patients' medical decisions.

BACKGROUND: Many decisions can be understood in terms of actors' valuations of benefits and costs. The article investigates whether this is also true of patient medical decision making. It aims to investigate (i) the importance patients attach to various reasons for and against nine medical decisions; (ii) how well the importance attached to benefits and costs predicts action or inaction; and (iii) how such valuations are related to decision confidence. METHODS: In a national random digit dial telephone survey of U.S. adults, patients rated the importance of various reasons for and against medical decisions they had made or talked to a health-care provider about during the past 2 years. Participants were 2575 English-speaking adults age 40 and older. Data were analysed by means of logistic regressions predicting action/inaction and linear regressions predicting confidence. RESULTS: Aggregating individual reasons into those that may be regarded as benefits and those that may be regarded as costs, and weighting them by their importance to the patient, shows the expected relationship to action. Perceived benefits and costs are also significantly related to the confidence patients report about their decision. CONCLUSION: The factors patients say are important in their medical decisions reflect a subjective weighing of benefits and costs and predict action/inaction although they do not necessarily indicate that patients are well informed. The greater the difference between the importance attached to benefits and costs, the greater patients' confidence in their decision.

Decision dissonance: evaluating an approach to measuring the quality of surgical decision making.

BACKGROUND: Good decision making has been increasingly cited as a core component of good medical care, and shared decision making is one means of achieving high decision quality. If it is to be a standard, good measures and protocols are needed for assessing the quality of decisions. Consistency with patient goals and concerns is one defining characteristic of a good decision. A new method for evaluating decision quality for major surgical decisions was examined, and a methodology for collecting the needed data was developed. METHODS: For a national probability sample of fee-for-service Medicare beneficiaries who had a coronary artery bypass graft (CABG), a lumpectomy or a mastectomy for breast cancer, or surgery for prostate cancer during the last half of 2008, a mail-survey of selected patients was carried out about one year after the procedures. Patients' goals and concerns, knowledge, key aspects of interactions with clinicians, and feelings about the decisions were assessed. A decision dissonance score was created that measured the extent to which patient ratings of goals ran counter to the treatment received. The construct and predictive validity of the decision dissonance score was then assessed. RESULTS: When data were averaged across all four procedures, patients with more knowledge and those who reported more involvement reported significantly lower Decision Dissonance Scores. Patients with lower Decision Dissonance Scores also reported more confidence in their decisions and feeling more positively about how the treatment turned out, and they were more likely to say that they would make the same decision again. CONCLUSIONS: Surveying discharged surgery patients is a feasible way to evaluate decision making, and Decision Dissonance appears to be a promising approach to validly measuring decision quality.

Do personal stories make patient decision aids more effective? A critical review of theory and evidence.

BACKGROUND: Patient decision aids support people to make informed decisions between healthcare options. Personal stories provide illustrative examples of others' experiences and are seen as a useful way to communicate information about health and illness. Evidence indicates that providing information within personal stories affects the judgments and values people have, and the choices they make, differentially from facts presented in non-narrative prose. It is unclear if including narrative communications within patient decision aids enhances their effectiveness to support people to make informed decisions. METHODS: A survey of primary empirical research employing a systematic review method investigated the effect of patient decision aids with or without a personal story on people's healthcare judgements and decisions. Searches were carried out between 2005-2012 of electronic databases (Medline, PsycINFO), and reference lists of identified articles, review articles, and key authors. A narrative analysis described and synthesised findings. RESULTS: Of 734 citations identified, 11 were included describing 13 studies. All studies found participants' judgments and/or decisions differed depending on whether or not their decision aid included a patient story. Knowledge was equally facilitated when the decision aids with and without stories had similar information content. Story-enhanced aids may help people recall information over time and/or their motivation to engage with health information. Personal stories affected both "system 1" (e.g., less counterfactual reasoning, more emotional reactions and perceptions) and "system 2" (e.g., more perceived deliberative decision making, more stable evaluations over time) decision-making strategies. Findings exploring associations with narrative communications, decision quality measures, and different levels of literacy and numeracy were mixed. The pattern of findings was similar for both experimental and real-world studies. CONCLUSIONS: There is insufficient evidence that adding personal stories to decision aids increases their effectiveness to support people's informed decision making. More rigorous research is required to elicit evidence about the type of personal story that a) encourages people to make more reasoned decisions, b) discourages people from making choices based on another's values, and c) motivates people equally to engage with healthcare resources.

Decision-making process reported by Medicare patients who had coronary artery stenting or surgery for prostate cancer.

BACKGROUND: Patients facing decisions should be told about their options, have the opportunity to discuss the pros and cons, and have their preferences reflected in the final decision. OBJECTIVES: To learn how decisions were made for two major preference-sensitive interventions. DESIGN: Mail survey of probability samples of patients who underwent the procedures. PARTICIPANTS: Fee-for-service Medicare beneficiaries who had surgery for prostate cancer or elective coronary artery stenting in the last half of 2008. MAIN MEASUREMENTS: Patients' reports of which options were presented for serious consideration, the amount of discussion of the pros and cons of the chosen option, and if they were asked about their preferences. RESULTS: The majority (64%) of prostate cancer surgery patients reported that at least one alternative to surgery was presented as a serious option. Almost all (94%) said they and their doctors discussed the pros, and 63% said they discussed the cons of surgery "a lot" or "some". Most (76%) said they were asked about their treatment preferences. Few who received stents said they were presented with options to seriously consider (10%). While most (77%) reported talking with doctors about the reasons for stents "a lot" or "some", few (19%) reported talking about the cons. Only 16% said they were asked about their treatment preferences. CONCLUSIONS: While prostate cancer surgery patients reported more involvement in decision making than elective stent patients, the reports of both groups document the need for increased efforts to inform and involve patients facing preference-sensitive intervention decisions.

Hospitalized patients' participation and its impact on quality of care and patient safety.

OBJECTIVE: To understand the extent to which hospitalized patients participate in their care, and the association of patient participation with quality of care and patient safety. DESIGN: Random sample telephone survey and medical record review. SETTING: US acute care hospitals in 2003. PARTICIPANTS: A total of 2025 recently hospitalized adults. MAIN OUTCOME MEASURES: Hospitalized patients reported participation in their own care, assessments of overall quality of care and the presence of adverse events (AEs) in telephone interviews. Physician reviewers rated the severity and preventability of AEs identified by interview and chart review among 788 surveyed patients who also consented to medical record review. RESULTS: Of the 2025 patients surveyed, 99.9% of patients reported positive responses to at least one of seven measures of participation. High participation (use of >4 activities) was strongly associated with patients' favorable ratings of the hospital quality of care (adjusted OR: 5.46, 95% CI: 4.15-7.19). Among the 788 patients with both patient survey and chart review data, there was an inverse relationship between participation and adverse events. In multivariable logistic regression analyses, patients with high participation were half as likely to have at least one adverse event during the admission (adjusted OR = 0.49, 0.31-0.78). CONCLUSIONS: Most hospitalized patients participated in some aspects of their care. Participation was strongly associated with favorable judgments about hospital quality and reduced the risk of experiencing an adverse event.

Validation of the SDM Process Scale to Evaluate Shared Decision-Making at Clinical Sites.

The Shared Decision-Making (SDM) Process scale (scored 0-4) uses 4 questions about decision-making behaviors: discussion of options, pros, cons, and preferences. We use data from mail surveys of patients who made surgical decisions at 9 clinical sites and a national web survey to assess the reliability and validity of the measure to assess shared decision-making at clinical sites. Patients at sites using decision aids to promote shared decision-making for hip, knee, back, or breast cancer surgery had significantly higher scores than national cross-section samples of surgical patients for 3 of 4 comparisons and significantly higher scores for both comparisons with "usual care sites." Reliability was supported by an intra-class correlation at the clinical site level of 0.93 and an average correlation of SDM scores for knee and hip surgery patients treated at the same sites of 0.56. The results document the reliability and validity of the measure to assess the degree of shared decision-making for surgical decisions at clinical sites.

Patient Preference Distribution for Use of Statin Therapy.

Importance: Thresholds for initiating statin therapy should be informed by patients' preferences. Objective: To define the preference distribution for statin therapy across the spectrum of cardiovascular disease (CVD) risk after participants were informed about the benefits and harms of statin therapy. Design, Setting, and Participants: A cross-sectional survey was conducted from May 13 to June 2, 2020. Participants included 304 individuals aged 40 to 75 years drawn from a nonprobability opt-in panel who had not taken a statin or proprotein convertase subtilisin/kexin type 9 inhibitor in the past 3 years and knew the results of their total cholesterol, high-density lipoprotein cholesterol, and blood pressure measurements. Exposures: Personalized 10-year CVD risk with and without statin therapy and potential harms of statins. Main Outcomes and Measures: The primary outcome was self-reported preference for statin therapy. Results: The 304 participants had a mean (SD) age of 54.8 (9.9) years; 152 were women (50.0%), 130 (42.8%) non-White, 50 (16.6%) had a high school degree or less education, and 153 (50.8%) reported never needing help reading health materials. When asked their preference for using statin therapy after reviewing their benefit and risk information, 45% of the participants reported they would definitely or probably choose statin therapy. As the risk increased, the proportion who would choose statin therapy generally increased (from 31.1% for a risk <5% to 82.6% for a risk >50%). The minimum risk threshold had to increase to 20% before 75% of respondents in that risk group would want statin therapy. For participants with a risk greater than 10%, the desire to use statin therapy decreased as participants' health literacy, subjective numeracy, and knowledge scores increased. Conclusions and Relevance: In this study, preferences for statin therapy for primary prevention of CVD appeared to vary across the spectrum of 10-year cardiovascular risk, but they were relatively flat at intermediate levels of risk. This preference distribution suggests a broad risk range for shared decision-making.

Development and Evaluation of the Shared Decision Making Process Scale: A Short Patient-Reported Measure.

BACKGROUND: The Shared Decision Making (SDM) Process scale is a short patient-reported measure of the amount of SDM that occurs around a medical decision. SDM Process items have been used previously in studies of surgical decision making and exhibited discriminant and construct validity. METHOD: Secondary data analysis was conducted across 8 studies of 11 surgical conditions with 3965 responses. Each study contained SDM Process items that assessed the discussion of options, pros and cons, and preferences. Item wording, content, and number of items varied, as did inclusion of measures assessing decision quality, decisional conflict (SURE scale), and regret. Several approaches for scoring, weighting, and the number of items were compared to identify an optimal approach. Optimal SDM Process scores were compared with measures of decision quality, conflict, and regret to examine construct validity; meta-analysis generated summary results. RESULTS: Although all versions of the scale were highly correlated, a short, partial credit, equally weighted version of the scale showed favorable properties. Overall, higher SDM Process scores were related to higher decision quality (d = 0.18, P = 0.029), higher SURE scale scores (d = 0.57, P < 0.001), and lower decision regret (d = -0.34, P < 0.001). Significant heterogeneity was present in all validity analyses. LIMITATIONS: Included studies all focused on surgical decisions, several had small sample sizes, and many were retrospective. CONCLUSION: SDM Process scores showed resilience to coding changes, and a scheme using the short, partial credit, with equal weights was adopted. The SDM Process scores demonstrated a small, positive relationship with decision quality and were consistently related to lower decision conflict and less regret, providing evidence of validity across several surgical decisions.

Validation of the 3-item What Engagement Looks Like (WELL) scale in patients with diabetes.

BACKGROUND: Patients' behaviors play a key role in chronic disease management, but how effective they are may depend on how engaged they feel. The objective was to develop a short measure of how much patients felt engaged in self-managing a chronic condition. Online test of a three-question series followed by a survey of physicians and their eligible diabetic patients. Physicians answered: 1) how well the physician thought the patient was managing his/her diabetes, and 2) how much effort the physician thought the patient was putting in. Each patient was mailed a survey that included three questions on self-management. Six hundred six patients from a national online consumer panel with diabetes or obesity, and 35 physicians from 3 primary care practices and a sample of 243 of their diabetic patients. Respondents were asked three questions about how much they thought their behavior could affect their health condition, how confident they were that they could do what was needed, and how involved they were in decisions about managing their condition. These items were summed to create a WELL score. Descriptive statistics and correlation coefficients were used to describe item relationships. Generalized Estimating Equations were used to predict how well the physician thought the patient was managing their diabetes and patient effort. RESULTS: Correlations among the three patient-reported items ranged from - 0.01 to 0.45. The WELL score was correlated with an existing measure of patient activation commitment (r = .43, p < 0.001) and found to be a significant predictor of physicians' ratings of how much effort patients devoted to condition management (b = 0.02, p = 0.001, OR = 1.02) after adjusting for confounders. The WELL score didn't predict physicians' ratings of how effective patients were (b = 0.003, p = .526, OR = 1.004) after their A1c score had been taken into account. CONCLUSION: Patients' WELL scores predicted physicians' ratings of patient effort in diabetes self-management.

National Survey of Decision-Making for Antidepressants and Educational Level.

BACKGROUND: Despite recommendations to screen adults for depression in primary care, little is known about how people across education levels decide to treat their depression and factors that influence their decision. METHODS: We conducted a secondary analysis of a national, probability-based web survey in English-speaking adults aged 40 or older living in the United States who reported they discussed starting or continuing an antidepressant with their clinician in the past 2 years. Respondents answered questions about knowledge, decision-making process, and demographics. Education level was analyzed using 5 ordered categories. The Shared Decision Making (SDM) Process score was used to assess patient involvement. Descriptive statistics, χ2 tests, analysis of variance, and regression models were used to describe the data and test associations. RESULTS: Of the 5682 people invited, 3396 answered questions about health decisions (59.8% response rate) and 385 reported discussing antidepressants. The mean percentage of knowledge questions answered correctly increased as education level increased (P = .008). The mean SDM Process score also increased with education (P = .001). There was an association between education and who made the treatment decision, suggesting that for respondents with less education, the clinician was more likely to decide (P = .001). Respondents with less education were less likely to report they would definitely make the same decision again (P = .000). CONCLUSIONS: Those with less education were even less informed, had lower SDM Process scores and were less likely to think they made the right decision about antidepressants. There is a need to ensure patients are better informed about and involved in treatment for depression.

Telephone health survey estimates: Effects of nonresponse and sample limitations.

OBJECTIVE: The objective of this study was to assess nonresponse error in telephone health survey data based on an address-based sample. DATA SOURCES: Telephone and in-person interviews in Greater Boston. STUDY DESIGN/DATA COLLECTION: Interviewers attempted telephone interviews at addresses that were matched to telephone numbers using questions drawn from federal health surveys. In-person household interviews were carried out with telephone nonrespondents and at addresses without matching telephone numbers. PRINCIPAL FINDINGS: After adjusting for demographic differences, only eight of 15 estimates based on the telephone interviews lay within two standard errors of the estimates when data from all three groups were included. CONCLUSIONS: For health surveys of address-based samples, many estimates based on telephone respondents differ from the total population in ways that cannot be corrected with simple demographic adjustments.

The effect of administration mode on CAHPS survey response rates and results: A comparison of mail and web-based approaches.

OBJECTIVE: The objective of this study was to compare response rates, respondents' characteristics, and substantive results for CAHPS surveys administered using web and mail protocols. DATA SOURCES: Patients who had one or more primary care visits in the preceding 6 months. STUDY DESIGN/DATA COLLECTION METHODS: Patients for whom primary care practices had email addresses were randomized to one of four survey administration protocols: web via a portal invitation; web via an email invitation; combination of web and mail; and mail only. Another sample of patients without known email addresses was surveyed by mail. Samples of nonrespondents to the Internet and mail protocols were surveyed by telephone. PRINCIPAL FINDINGS: Response rates to surveys administered using the Internet protocols were lower than for the surveys administered by mail (20 percent vs over 40 percent). However, characteristics of respondents and survey answers were very similar across protocols. Respondents without email addresses were older, less educated, and more likely to be male than those with email addresses, and there were a few differences in their responses. There was little evidence of nonresponse bias in either the mail or web protocols. CONCLUSION: In this well-educated patient population, web protocols had lower response rates, but substantive results very similar to those from mail protocols.