Ethics and methods in surgical trials.

This paper focuses on invasive therapeutic procedures, defined as procedures requiring the introduction of hands, instruments, or devices into the body via incisions or punctures of the skin or mucous membranes performed with the intent of changing the natural history of a human disease or condition for the better. Ethical and methodological concerns have been expressed about studies designed to evaluate the effects of invasive therapeutic procedures. Can such studies meet the same standards demanded of those, for example, evaluating pharmaceutical agents? This paper describes a research project aimed at examining the interplay and sometimes apparent conflict between ethical standards for human research and standards for methodological rigor in trials of invasive procedures. The paper discusses how the authors plan to develop a set of consensus standards that, if met, would result in substantial and much-needed improvements in the methodological and ethical quality of such trials.

Using the national registry of HIV-infected veterans in research: lessons for the development of disease registries.

Disease-specific registries have many important applications in epidemiologic, clinical and health services research. Since 1989 the Department of Veterans Affairs has maintained a national HIV registry. VA's HIV registry is national in scope, it contains longitudinal data and detailed resource utilization and clinical information. To describe the structure, function, and limitations of VA's national HIV registry, and to test its accuracy and completeness. The VA's national HIV registry contains data that are electronically extracted from VA's computerized comprehensive clinical and administrative databases, called Veterans Integrated Health Systems Technology and Architecture (VISTA). We examined the number of AIDS patients and the number of new patients identified to the registry, by year, through December 1996. We verified data elements against information obtained from the medical records at five VA sites. By December 1996, 40,000 HIV-infected patients had been identified to the registry. We encountered missing data and problems with data classification. Missing data occurred for some elements related to the computer programming that creates the registry (e.g., pharmacy files), and for other elements because manual entry is required (e.g., ethnicity). Lack of a standardized data classification system was a problem, especially for the pharmacy and laboratory files. In using VA's national HIV registry we have learned important lessons, which, if taken into account in the future, could lead to the creation of model disease-specific registries.

A psychometric analysis of the measurement level of the rating scale, time trade-off, and standard gamble.

A fundamental assumption of utility-based analyses is that patient utilities for health states can be measured on an equal-interval scale. This assumption, however, has not been widely examined. The objective of this study was to assess whether the rating scale (RS), standard gamble (SG), and time trade-off (TTO) utility elicitation methods function as equal-interval level scales. We wrote descriptions of eight prostate-cancer-related health states. In interviews with patients who had newly diagnosed, advanced prostate cancer, utilities for the health states were elicited using the RS, SG, and TTO methods. At the time of the study, 77 initial and 73 follow-up interviews had. been conducted with a consecutive sample of 77 participants. Using a Rasch model, the boundaries (Thurstone Thresholds) between four equal score sub-ranges of the raw utilities were mapped onto an equal-interval logit scale. The distance between adjacent thresholds in logit units was calculated to determine whether the raw utilities were equal-interval. None of the utility scales functioned as interval-level scales in our sample. Therefore, since interval-level estimates are assumed in utility-based analyses, doubt is raised regarding the validity of findings from previous analyses based on these scales. Our findings need to be replicated in other contexts, and the practical impact of non-interval measurement on utility-based analyses should be explored. If cost-effectiveness analyses are not found to be robust to violations of the assumption that utilities are interval, serious doubt will be cast upon findings from utility-based analyses and upon the wisdom of expending millions in research dollars on utility-based studies.

SODA (severity of dyspepsia assessment): a new effective outcome measure for dyspepsia-related health.

The aim of this research was to develop and evaluate an instrument for measuring dyspepsia-related health to serve as the primary outcome measure for randomized clinical trials. Building on our previous work we developed SODA (Severity of Dyspepsia Assessment), a multidimensional dyspepsia measure. We evaluated SODA by administering it at enrollment and seven follow-up visits to 98 patients with dyspepsia who were randomized to a 6-week course of omeprazole versus placebo and followed over 1 year. The mean age was 53 years, and six patients (6%) were women. Median Cronbach's alpha reliability estimates over the eight visits for the SODA Pain Intensity, Non-Pain Symptoms, and Satisfaction scales were 0.97, 0.90, and 0.92, respectively. The mean change scores for all three scales discriminated between patients who reported they were improved versus those who were unchanged, providing evidence of validity. The effect sizes for the Pain Intensity (.98) and Satisfaction (.87) scales were large, providing evidence for responsiveness. The effect size for the Non-Pain Symptoms scale was small (.24), indicating lower responsiveness in this study sample. SODA is a new, effective instrument for measuring dyspepsia-related health. SODA is multidimensional and responsive to clinically meaningful change with demonstrated reliability and validity.

When does regionalization of expensive medical care save money?

Regionalization of expensive, high-technology medical care is often proposed as a way to reduce medical costs. Most empirical estimates of the cost implications of regionalization suffer from methodological shortcomings. Here, we discuss all the factors that must be taken into account to produce an accurate assessment of how regionalization changes costs. These factors include the following: (1) The extent of resource sharing among different services; (2) The extent of unused capacity; (3) Whether regionalized facilities have high, low or average costs; (4) Costs of a regionalized system, including transporting patients to the regionalized facilities, coordinating care between the referring and regionalized providers, and out-of network care; (5) The effect of regionalization on the volume of care; and (6) whether a short- or long-term view is taken.

Living with treatment decisions: regrets and quality of life among men treated for metastatic prostate cancer.

PURPOSE: To examine variation in men's long-term regret of treatment decisions, ie, surgical versus chemical castration, for metastatic prostate cancer and its associations with quality of life. METHODS: Survey of previously treated patients to assess treatment decisions and quality of life, supplemented with focus groups. Two items addressing whether a patient wished he could change his mind and the belief that he would have been better off with the treatment not chosen were combined in classifying survey respondents as either satisfied or regretful. Chi(2) and t tests were used to test associations between regret and treatment history, complications, and quality of life. RESULTS: Survey respondents included 201 men aged 45 to 93 years (median, 71 years), who had begun treatment (71% chemical castration, 29% orchiectomy) a median of 2 years previously. Most reported complications: hot flashes (70%), nausea (34%), and erectile dysfunction (81%). Most were satisfied with the treatment decision, but 23% expressed regret. Regretful men more frequently reported surgical (43%) versus chemical (36%) castration (P: = .030) and nausea in the past week (54% v 32%; P: = .010) but less frequently reported erectile dysfunction (56% v 72%; P: = .048). Regretful men indicated poorer scores on every measure of generic and prostate cancer-related quality of life. Qualitative analyses revealed substantial uncertainty about the progress of their disease and the quality of the decisions in which patients participated. CONCLUSION: Regret was substantial and associated with treatment choice and quality of life. It may derive from underlying psychosocial distress and problematic communication with physicians when decisions are being reached and over subsequent years.

A national survey of policies on disclosure of conflicts of interest in biomedical research.

BACKGROUND: Conflicts of interest pose a threat to the integrity of scientific research. The current regulations of the U.S. Public Health Service and the National Science Foundation require that medical schools and other research institutions report the existence of conflicts of interest to the funding agency but allow the institutions to manage conflicts internally. The regulations do not specify how to do so. METHODS: We surveyed all medical schools (127) and other research institutions (170) that received more than $5 million in total grants annually from the National Institutes of Health or the National Science Foundation; 48 journals in basic science and clinical medicine; and 17 federal agencies in order to analyze their policies on conflicts of interest. RESULTS: Of the 297 institutions, 250 (84 percent) responded by March 2000, as did 47 of the 48 journals and 16 of the 17 federal agencies. Fifteen of the 250 institutions (6 percent)--5 medical schools and 10 other research institutions--reported that they had no policy on conflicts of interest. Among the institutions that had policies, there was marked variation in the definition and management of conflicts. Ninety-one percent had policies that adhered to the federal threshold for disclosure ($10,000 in annual income or equity in a relevant company or 5 percent ownership), and 9 percent had policies that exceeded the federal guidelines. Only 8 percent had policies requiring disclosure to funding agencies, only 7 percent had such policies regarding journals, and only 1 percent had policies requiring the disclosure of information to the relevant institutional review boards or to research subjects. Twenty journals (43 percent) reported that they had policies requiring disclosure of conflicts of interest. Only four federal agencies had policies that explicitly addressed conflicts of interest in extramural research, and all but one of the agencies relied primarily on institutional discretion. CONCLUSIONS: There is substantial variation among policies on conflicts of interest at medical schools and other research institutions. This variation, combined with the fact that many scientific journals and funding agencies do not require disclosure of conflicts of interest, suggests that the current standards may not be adequate to maintain a high level of scientific integrity.

A trial for comparing methods for eliciting treatment preferences from men with advanced prostate cancer: results from the initial visit.

OBJECTIVE: The objective of this study was to evaluate the convergent validity of 3 types of utility measures: standard gamble, time tradeoff, and rating scale. RESEARCH DESIGN: A prospective cohort of 120 men with advanced prostate cancer were first asked to rank order 8 health states, and then utility values were obtained from each participant for each of the 8 health states through 2 of the 3 techniques evaluated (standard gamble, time tradeoff and rating scale). Participants were randomly assigned to 1 of 3 possible pairs of techniques. The validity of the 3 methods, as measured by the convergence and raw score differences of the techniques, was assessed with ANOVA. The ability of the techniques to differentiate health states was determined. The inconsistencies between rankings and utility values were also measured. Proportions of illogical utility responses were assessed as the percent of times when states with more symptoms were given higher or equal utility values than states with fewer symptoms. RESULTS: There were significant differences in raw scores between techniques, but the values were correlated across health states. Utility values were often inconsistent with the rank order of health states. In addition, utility assessment did not differentiate the health states as well as the rank order. Furthermore, utility values were often illogical in that states with more symptoms received equal or higher utility values than states with fewer symptoms. CONCLUSIONS: Use of the utility techniques in cost-effectiveness analysis and decision making has been widely recommended. The results of this study raise serious questions as to the validity and usefulness of the measures.

Clinical and socioeconomic determinants of health care use among HIV-infected patients in the Department of Veterans Affairs.

This study estimates the impact of clinical and socioeconomic characteristics on health care use for HIV-infected patients. Data come from the Department of Veterans Affairs (VA) HIV Registry, which electronically extracts data from patients' automated medical records, and from patient interviews. Unlike prior studies, this analysis includes a staging system incorporating CD4 count and AIDS-defining diagnoses. Results showed that clinical factors were the most important determinants of health care use; socioeconomic variables were seldom significant. These findings were expected, since the VA is an equal access system, providing care regardless of socioeconomic status.

Veterans Affairs Quality Enhancement Research Initiative in chronic heart failure.

Chronic heart failure (CHF) is a highly prevalent condition associated with serious morbidity, intense levels of health services use, and shortened survival. It is also a condition for which ameliorative therapies exist. The evidence indicates that there is substantial need to change clinical practice and health care delivery for people with CHF and thereby improve their outcomes. The goal of the Veterans Affairs (VA) Quality Enhancement Research Initiative in CHF (CHF QUERI) is to create measurable, rapid, and sustainable improvements in quality of care and health outcomes of veterans with heart failure. This article describes the current state of knowledge and practice in care for people with CHF. Using the framework of the 5 steps of the QUERI process, we point out the gaps in research and practice that must be filled if the CHF QUERI is to achieve its goal. We relate our recommendations for how the VA can put its research and administrative infrastructure to work to fill the gaps. Lessons learned about CHF in the course of the CHF QUERI will be applicable to all people with heart failure and to all health care systems--VA as well as non-VA--that care for them.

The health status of veterans using mobile clinics in rural areas.

Between 1992 and 1994, the Department of Veterans Affairs (VA) experimented with mobile clinics to provide health care for rural veterans. The objective was to assess the health status of rural mobile clinics' patients and compare this with patients receiving care in VA hospital-based clinics. This study hypothesized that hospital-based clinic patients would be more ill (i.e., have a greater reduction in health status). The Medical Outcomes Study (MOS) Short Form was used to evaluate patients' health status. Most patients sought care for the management of chronic disease. Patients in both groups had similar types of diseases. Mobile clinic patients were as ill as hospital-based patients (i.e., similar health status scores). This study shows that rural veterans have a case mix and a reduction in health status similar to that of VA hospital-based patients. Planners should account for this health reduction when planning the kinds of facilities and services needed in rural areas.

Rates of health services utilization and survival in patients with heart failure in the Department of Veterans Affairs medical care system.

The objective of this study was to describe patterns of hospital and clinic use and survival for a large nationwide cohort of patients with heart failure. A retrospective cohort study of patients treated in the Veterans Affairs medical care system was conducted using linked administrative databases as data sources. In 1996, the average heart failure cohort member had 1-2 hospitalizations, 14 inpatient days, 6-7 visits with the primary physician, 15 other visits for consultations or tests, and 1-2 urgent care visits per 12 months. The overall risk-adjusted 5-year survival rate was 36%. Hospital use rates in the cohort fell dramatically between 1992 and 1996. One-year survival rates increased slightly over the period. Patients with heart failure are heavy users of services and have a very poor prognosis. Utilization and outcome data indicate the need for major efforts to assure quality of care and to devise innovative ways of delivering comprehensive services.

How good are US studies of HIV costs of care?

BACKGROUND: Valid, timely estimates of the costs of HIV care are needed by health planners and policy makers. OBJECTIVE: To perform a methodologic critique of published estimates of resource utilization and costs of HIV care. DATA SOURCES: MEDLINE database for 1990-1998. DATA SELECTION: Included articles focused on adults with a spectrum of HIV disease in which the authors developed their own resource use and cost data. Thirty one articles met these criteria. DATA EXTRACTION: Studies were compared based on: (1) utilization and cost estimates, in 1995 dollars; (2) study period; (3) research design; (4) sampling frame; (5) sample size and patient characteristics; (6) data sources and scope of services; and (7) methods used in the analysis. DATA SYNTHESIS: The most recent estimates pertain to the first half of 1995, before the use of protease inhibitor therapy. We found wide variations in the estimates and identified three major sources for this: (1) patient samples that were restricted to subgroups of the national HIV-infected population; (2) utilization data that were limited in scope (e.g., inpatient care only); and (3) invalid methods for estimating annual or lifetime costs, particularly in dealing with decedents. CONCLUSIONS: To accurately estimate resource use and costs for HIV care nationwide, a nationally representative probability sample of HIV-infected patients is required. Even in research that is not intended to provide national estimates, the scope of utilization data should be broadened and greater attention to methodologic issues in the analysis of annual and lifetime costs is needed.

An empirical assessment of the validity of explicit and implicit process-of-care criteria for quality assessment.

OBJECTIVE: To evaluate the validity of three criteria-based methods of quality assessment: unit weighted explicit process-of-care criteria; differentially weighted explicit process-of-care criteria; and structured implicit process-of-care criteria. METHODS: The three methods were applied to records of index hospitalizations in a study of unplanned readmission involving roughly 2,500 patients with one of three diagnoses treated at 12 Veterans Affairs hospitals. Convergent validity among the three methods was estimated using Spearman rank correlation. Predictive validity was evaluated by comparing process-of-care scores between patients who were or were not subsequently readmitted within 14 days. RESULTS: The three methods displayed high convergent validity and substantial predictive validity. Index-stay mean scores, using explicit criteria, were generally lower in patients subsequently readmitted, and differences between readmitted and nonreadmitted patients achieved statistical significance as follows: mean readiness-for-discharge scores were significantly lower in patients with heart failure or with diabetes who were readmitted; and mean admission work-up scores were significantly lower in patients with lung disease who were readmitted. Scores derived from the structured implicit review were lower in patients eventually readmitted but significantly so only in diabetics. CONCLUSIONS: These three criteria-based methods of assessing process of care appear to be measuring the same construct, presumably "quality of care." Both the explicit and implicit methods had substantial validity, but the explicit method is preferable. In this study, as in others, it had greater inter-rater reliability.

The hospital multistay rate as an indicator of quality of care.

OBJECTIVES: To evaluate the hospital multistay rate to determine if it has the attributes necessary for a performance indicator that can be applied to administrative databases. DATA SOURCES/STUDY SETTING: The fiscal year 1994 Veterans Affairs Patient Treatment File (PTF), which contains discharge data on all VA inpatients. STUDY DESIGN: Using a retrospective study design, we assessed cross-hospital variation in (a) the multistay rate and (b) the standardized multistay ratio. A hospital's multistay rate is the observed average number of hospitalizations for patients with one or more hospital stays. A hospital's standardized multistay ratio is the ratio of the geometric mean of the observed number of hospitalizations per patient to the geometric mean of the expected number of hospitalizations per patient, conditional on the types of patients admitted to that hospital. DATA COLLECTION/EXTRACTION METHODS: Discharge data were extracted for the 135,434 VA patients who had one or more admissions in one of seven disease groups. PRINCIPAL FINDINGS: We found that 17.3 percent (28,300) of the admissions in the seven disease categories were readmissions. The average number of stays per person (multistay rate) for an average of seven months of follow-up ranged from 1.15 to 1.45 across the disease categories. The maximum standardized multistay ratio ranged from 1.12 to 1.39. CONCLUSIONS: This study has shown that the hospital multistay rate offers sufficient ease of measurement, frequency, and variation to potentially serve as a performance indicator.

Assessing the performance of utility techniques in the absence of a gold standard.

BACKGROUND: Utility techniques are the most commonly used means to assess patient preferences for health outcomes. However, whether utility techniques produce valid measures of preference has been difficult to determine in the absence of a gold standard. OBJECTIVE: To introduce and demonstrate two methods that can be used to evaluate how well utility techniques measure patients' preferences. SUBJECTS AND DESIGN: Patients treated for advanced prostate cancer (n = 57) first ranked eight health states in order of preference. Four utility techniques were then used to elicit patients' utilities for each health state. MEASURES: The rating scale, standard gamble, time trade-off, and a modified version of willingness-to-pay techniques were used to elicit patients' utilities. Technique performance was assessed by computing a differentiation and inconsistency score for each technique. RESULTS: Differentiation scores indicated the rating scale permitted respondents to assign unique utility values to about 70% of the health states that should have received unique values. When the other techniques were used, about 40% or less of the health states that should have received unique utility scores actually did receive unique utility scores. Inconsistency scores, which indicate how often participants assign utility scores that contradict how they value health states, indicated that the willingness-to-pay technique produced the lowest rate of inconsistency (10%). However, this technique did not differ significantly from the rating scale or standard gamble on this dimension. CONCLUSIONS: Differentiation and inconsistency offer a means to evaluate the performance of utility techniques, thereby allowing investigators to determine the extent to which utilities they have elicited for a given decision problem are valid. In the current investigation, the differentiation and inconsistency methods indicated that all four techniques performed at sub-optimal levels, though the rating scale out-performed the standard gamble, time trade-off, and willingness-to-pay techniques.

Evaluation of the VA mobile clinics demonstration project.

In 1988 the Veterans' Benefits and Services Act attempted to solve the problem of the lack of adequate VA healthcare facilities in rural areas by establishing a demonstration program using mobile clinics. Six clinics operated in areas that were at least 100 miles from a VA healthcare facility during the time period between October 1, 1992 and May 28, 1994. This article evaluated the effect of the mobile clinics' structural limitations on clinical care, the increased number of sites on VA usage, and cost. Limited space for storage of medical records and the unavailability of laboratory, electrocardiographic, or radiographic facilities significantly affected clinical practice. However, even with these space limitations, veterans' use of healthcare in the areas served by the mobile clinics increased significantly in comparison to reference areas. The direct costs per visit averaged more than three times what the VA would have reimbursed the private sector.

Evaluation of a multidimensional measure of dyspepsia-related health for use in a randomized clinical trial.

In previous work, we developed a multidimensional measure of dyspepsia-related health. To evaluate the adequacy of this instrument as an outcome measure for a large-scale, multicenter, randomized clinical trial, we used Rasch analysis to address three questions: (1) Are the scales interval-level? (2) Do the scales measure precisely across the entire range of dyspepsia outcomes? (3) Do the scales' items have an optimal number of response categories? We found that the scales were not interval-level and that they did not measure effectively at low or high levels of the dyspepsia-related outcomes. Our results also suggest that patients were capable of discriminating among only four- to seven-item response categories. Further studies are needed to identify items that effectively measure high and low levels of dyspepsia-related outcomes and to validate that decreasing the number of response categories improves the psychometric properties of these scales.

Cost implications of regionalizing open heart surgery units.

This study calculated the potential change in costs from regionalizing open heart surgery units in a geographic network of the Department of Veterans Affairs (VA). It used data from the VA's cost accounting system, and the authors conducted a sensitivity analysis. Under consolidation, savings from closing an open heart surgery unit would be partially offset by the costs of treating nonemergency cases at other VAs, treating emergency cases at non-VA hospitals, and transporting patients to regionalized facilities. Nevertheless, the potential savings from consolidation would exceed $3 million, or 18% of the network's costs of treating open heart surgery patients.

Use of a cost accounting system to evaluate costs of a VA special program.

BACKGROUND: The Department of Veterans Affairs (VA) established six mobile clinics to provide care for rural veterans. Each was operated by a parent VA Medical Center (VAMC). OBJECTIVE: To describe the use of a cost-accounting system which does not provide costs at the service or patient level to determine the costs of the mobile clinics. RESEARCH DESIGN: Costs per visit were compared among the mobile clinics with the parent VAMCs and with simulated fixed-location clinics. Cost data came from VA's Centralized Accounting for Local Management (CALM) data. Utilization data came from VA's outpatient file. RESULTS: Information was obtained from the VAMCs' fiscal services to reallocate costs among the CALM subaccounts to generate cost data that was comparable among the mobile clinics. Costs per visit for the mobile clinics were twice as high as those of the parent VAMCs. Costs per visit would be lower at fixed-location clinics unless the volume were substantially less than that provided by the mobile clinics. CONCLUSION: Differences between cost allocations for accounting purposes and research are likely to necessitate adjusting cost accounting data for research purposes. Fortunately, information from the accountants or primary data can lead to a cost database which is appropriate for research evaluations. In the mobile clinics study, the analysis of cost accounting data led to the conclusion that mobile clinics were not a cost-effective way in which to provide care to rural veterans.