Toward a national consumer survey: evaluation of the CABHS and MHSIP instruments.

This article describes a study evaluating the Consumer Assessment of Behavioral Health Survey (CABHS) and the Mental Health Statistics Improvement Program (MHSIP) surveys. The purpose of the study was to provide data that could be used to develop recommendations for an improved instrument. Subjects were 3,443 adults in six behavioral health plans. The surveys did not differ significantly in response rate or consumer burden. Both surveys reliably assessed access to treatment and aspects of appropriateness and quality. The CABHS survey reliably assessed features of the insurance plan; the MHSIP survey reliably assessed treatment outcome. Analyses of comparable items suggested which survey items had greater validity. Results are discussed in terms of consistency with earlier research using these and other consumer surveys. Implications and recommendations for survey development, quality improvement, and national policy initiatives to evaluate health plan performance are presented.

Use of consumer ratings for quality improvement in behavioral health insurance plans.

BACKGROUND: The Consumer Assessment of Behavioral Healthcare Services (CABHS) survey collects consumers' reports about their health care plans and treatment. The use of the CABHS to identify opportunities for improvement, with specific attention to how organizations have used the survey information for quality improvement, is described. METHODS: In 1998 and 1999, data were collected from five groups of adult patients in commercial health plans and five groups of adult patients in public assistance health plans with services received through four organizations (one of three managed behavioral health care organizations or a health system). Patients who received behavioral health care services during the previous year were mailed the CABHS survey. Non-respondents were contacted by telephone to complete the survey. RESULTS: Response rates ranged from 49% to 65% for commercial patient groups and from 36% to 51% for public assistance patients. Promptly getting treatment from clinicians and aspects of care most influenced by health plan policies and operations, such as access to treatment and plan administrative services, received the least positive responses, whereas questions about communication received the most positive responses. In addition, questions about access- and plan-related aspects of quality showed the most interplan variability. Three of the organizations in this study focused quality improvement efforts on access to treatment. DISCUSSION: Surveys such as the CABHS can identify aspects of the plan and treatment that are improvement priorities. Use of these data is likely to extend beyond the behavioral health plan to consumers, purchasers, regulators, and policymakers, particularly because the National Committee for Quality Assurance is encouraging behavioral health plans to use a similar survey for accreditation purposes.

Respondent selection by mail: obtaining probability samples of health plan enrollees.

OBJECTIVES: This study was done to assess the feasibility of respondent selection by mail to obtain random samples of both child and adult enrollees of health plans when only subscriber contact information is available. METHODS: The subjects were enrollees of health plans covered under the policies of employees of the State of Washington. Subscribers were eligible for inclusion in the study samples if they had been enrolled for at least 6 months and, depending on the test, had at least one child and/or a spouse enrolled under their policy. Subjects were randomly assigned to six groups: three approaches for sampling children, two approaches for sampling adults, and one test of parents' willingness to return a questionnaire about themselves. Child selection protocols involved asking respondents to complete either a child-only or a dual questionnaire, asking them to follow a decision rule to choose a sample person, and collecting data in two phases, asking respondents to return material twice. RESULTS: Results indicated that asking subscribers to select an adult respondent by mail was not a success. At least given the procedures we used, adults did not demonstrate good compliance with the respondent selection process offered them. In contrast, parents proved willing to follow a more complicated child selection rule and to do it nearly perfectly. CONCLUSIONS: Parents will follow a decision rule to select an eligible child, but requiring this additional respondent selection step may be associated with a slightly decreased response rate. Asking parents to return materials twice is feasible, but it is too cumbersome a procedure to be practical. It is possible to collect data about both an adult and a child using a dual instrument; however, the increase in information is tempered by a decrease in response rates.