Problems in the conduct and analysis of randomized clinical trials. Are we getting the right answers to the wrong questions?

To achieve the goal of validity, the randomized clinical trial has emerged as the scientific "gold standard" for evaluating therapies in clinical medicine. Regardless of how well randomized clinical trials are designed, however, problems often occur during the conduct of the trials that give rise to methodologic challenges in the analysis of results. Primarily two types of problems, changes in intended treatment and the failure to ascertain the study outcomes, occur during the conduct of randomized clinical trials. We studied the current analytic strategies that are used to deal with these problems and how the use of these analytic strategies can change the focus of the research so that the trial no longer answers the relevant question. To ensure that the right question is answered, new methods of design and analysis are required that balance the goals of validity and clinical pertinence.

An ecogenetic hypothesis for lung cancer in women.

Epidemiologic studies have established a strong association between cigarette smoking and lung cancer, but the risk estimates for women are less impressive than for men. We assessed the possible role of family history and its interrelationship with cigarette smoking as risk factors for lung cancer in women by conducting a case-control study. Among 112 cases, 7% had a primary family member with lung cancer compared with only 3% of 224 controls for an odds ratio of 2.8. Cigarette smoking was present for 87% of the cases and 41% of the controls for an odds ratio of 11.3. The ecogenetic interrelationship of cigarette smoking and family history was supported by the gradient in the odds ratio for lung cancer created by the two variables: patients who never smoked but had a positive family history had an odds ratio of 5.7; patients who smoked but had a negative family history had an odds ratio of 15.1; and patients who smoked and had a positive family history had an odds ratio of almost 30. We conclude that family history may be an important risk factor for lung cancer in women, and that the ecogenetic interrelationship of family history with cigarette smoking may help explain the occurrence of this disease in women.

Zidovudine (AZT) for treatment of patients infected with human immunodeficiency virus type 1. An evaluation of effectiveness in clinical practice.

To learn about the patterns of use and the effectiveness of zidovudine therapy in clinical practice, we conducted an observational cohort study of 86 patients with human immunodeficiency virus type 1 infection. All patients were followed up for at least 6 months after starting zidovudine (AZT) therapy. Of the 86 patients, 78 (91%) initially received full-dosage zidovudine (1200 mg/d), and eight received a reduced dosage (600 mg/d). During follow-up, the number able to maintain full-dosage zidovudine therapy decreased to 54 (63%) at 3 months and 40 (47%) at 6 months. Thirty-five patients required dosage reductions that lasted at least 7 days and were not preceded by an adverse outcome (death or opportunistic infection). Overall, adverse outcomes occurred for nine (26%) of those with dosage reductions compared with 22 (43%) of 51 patients with no previous dosage change. Even after adjusting for baseline cytopenias and the time of the dosage reductions, adverse outcomes did not occur significantly more often in patients who received reductions in their zidovudine dosage. Our results indicate that full-dosage zidovudine therapy cannot be maintained for most patients infected with human immunodeficiency virus, but that clinicians need not be pessimistic about treatment outcomes when dosage reductions are needed.

A quality-of-life instrument for young hemorrhagic stroke patients.

BACKGROUND AND PURPOSE: Hemorrhagic stroke has a high initial mortality rate. While survivors often recover motor function, many experience significant changes in their quality of life (QOL). Available outcome measures assess neurological impairment, disability, or handicap, yet often inadequately characterize the full impact of a stroke on patients' lives. In this study, we develop and validate a QOL instrument specific for young patients with hemorrhagic strokes. METHODS: Methodological guidelines for instrument development were initially established. Based on the content of 40 open-ended patient interviews, a 54-item instrument (HSQuale) was developed. The reliability (test-retest and internal consistency) and validity (content and construct) of HSQuale were assessed in another 71 patients (18 to 49 years of age, 63% women, 77% white), at 1 year after their hemorrhagic stroke. Comparisons were made between HSQuale and other commonly used outcome measures. RESULTS: HSQuale demonstrated reproducibility (test-retest kappa, 0.40 to 0.96) and internal consistency (Cronbach alpha >/=0.80 for 5 of 7 domains). HSQuale scores had broad frequency distributions (

Insulin resistance and risk for stroke.

BACKGROUND AND PURPOSE: Resistance to insulin-mediated glucose uptake by peripheral tissues is a cardinal defect in type 2 diabetes mellitus. Insulin resistance is also common among nondiabetic individuals, and may be an important risk factor for stroke in both populations. The authors review the definition, epidemiology, and treatment of insulin resistance. METHODS: The authors searched Medline (1977-2001) and reviewed bibliographies to identify pertinent English-language publications. RESULTS: Insulin resistance is present in most patients with type 2 diabetes. It is also common among elderly persons, certain ethnic groups, and persons with hypertension, obesity, physical deconditioning, and vascular disease. The principal pathophysiologic defect is impaired intracellular signaling in muscle tissue leading to defective glycogen synthesis. Insulin resistance is associated with numerous metabolic, hematologic, and cellular events that promote atherosclerosis and coagulation. The association between insulin resistance and risk for stroke has been examined in four case-control studies and five prospective observational cohort studies. Six of the nine studies are methodologically sound and provide evidence that insulin resistance is associated with risk for stroke. CONCLUSION: Insulin resistance may be a prevalent risk factor for stroke. New drugs can safely reduce insulin resistance and may have a role in stroke prevention.

Patient acceptance of influenza vaccination.

PURPOSE: To determine whether patients' demographic, medical, and personal characteristics, including attitudes and beliefs about vaccination, health, and medical providers, are associated with acceptance of influenza vaccine. PATIENTS AND METHODS: Nine hundred sixty-five patients attending a university hospital-based general medicine clinic during the fall influenza vaccination period, including 624 patients for whom influenza vaccine was indicated, were observed in a prospective cohort study. In addition, 58 patients who refused influenza vaccine and an equal number who accepted it were interviewed over the telephone to examine their beliefs and behaviors in greater detail. RESULTS: Seventy-five percent of patients for whom influenza vaccine was indicated received it. Prospectively assessed patient characteristics that were significantly associated with nonvaccination included not believing vaccine prevents "flu" (relative risk [RR] 5.3), never received pneumococcal vaccine (RR 3.5), not vaccinated against influenza the previous year (RR 3.5), never vaccinated against influenza (RR 2.3), and felt sick after previous influenza vaccination (RR 2.3). Demographic characteristics and medical diagnoses were not significantly related to vaccination. Almost one half of 58 interviewed subjects who refused influenza vaccine cited fear of a reaction. Among retrospectively determined attitudes and beliefs significantly associated with refusal of influenza vaccine were not believing the vaccine works well (odds ratio [OR] 11.6), concern about a reaction (OR 9.3), and perception that the medical provider had not recommended it (OR 5.8). CONCLUSION: Demographic characteristics of patients and their medical diagnoses were not associated with acceptance of influenza vaccination. Among patients who were not vaccinated, doubts about the efficacy of influenza vaccine and fear of its side effects were common, and their perceptions of the medical provider's recommendation of vaccine appeared to be an important factor in the decision whether to accept it.

Developing improved observational methods for evaluating therapeutic effectiveness.

Therapeutic efficacy is often studied with observational surveys of patients whose treatments were selected nonexperimentally. The results of these surveys are distrusted because of the fear that biased results occur in the absence of experimental principles, particularly randomization. The purpose of the current study was to develop and validate improved observational study designs by incorporating many of the design principles and patient assembly procedures of the randomized trial. The specific topic investigated was the prophylactic effectiveness of beta-blocker therapy after an acute myocardial infarction. To accomplish the research objective, three sets of data were compared. First, we developed a restricted cohort based on the eligibility criteria of the randomized clinical trial; second, we assembled an expanded cohort using the same design principles except for not restricting patient eligibility; and third, we used the data from the Beta Blocker Heart Attack Trial (BHAT), whose results served as the gold standard for comparison. In this research, the treatment difference in death rates for the restricted cohort and the BHAT trial was nearly identical. In contrast, the expanded cohort had a larger treatment difference than was observed in the BHAT trial. We also noted the important and largely neglected role that eligibility criteria may play in ensuring the validity of treatment comparisons and study outcomes. The new methodologic strategies we developed may improve the quality of observational studies and may be useful in assessing the efficacy of the many medical/surgical therapies that cannot be tested with randomized clinical trials.

The evolving clinical status of patients after a myocardial infarction: the importance of post-hospital data for mortality prediction.

Studies predicting mortality after myocardial infarction (MI) usually rely on in-hospital data, and combine patients admitted for the first MI with recurrent MI patients. Since treatment decisions are often made or modified at the first outpatient clinic visit, this study was designed to evaluate the importance of post-hospital data on mortality prediction after a first myocardial infarction (MI). An inception cohort of patients enrolled in the Beta-Blocker in Heart Attack Trial (n = 2830) was included. Forty-three variables (including in-hospital and post-hospital data) were evaluated using stepwise logistic regression. Ten variables were independently associated with 1-year mortality: five used in-hospital data (history of hypertension, hypercholesterolemia, congestive heart failure [CHF], ventricular tachycardia, and age); and five variables depended on post-hospital data collected at the first outpatient visit (CHF after discharge, New York Heart Association functional class, heart rate, pulmonary rates, and smoking). Two predictive systems were developed that partitioned patients into one of four classes with distinct mortality risks: a composite system using the 10 in- and post-hospital variables, and a system using only the 5 in-hospital variables. Mortality risk for the composite system classes ranged from 0.6 to 20.0% (I [n = 861], 0.6%; II [n = 1151], 2.3%; III [n =698], 4.3%; IV [n = 120], 20.0%). In contrast, the range of mortality risk using the in-hospital data only system was less (1 to 8.3%). Most importantly, a distinct gradient within each class of the in-hospital data only system was created by the addition of the post-hospital data. This study demonstrates that risk stratification after an acute first MI is improved by the addition of post-hospital data.

Can treatment that is helpful on average be harmful to some patients? A study of the conflicting information needs of clinical inquiry and drug regulation.

Randomized controlled trials are conducted with heterogeneous groups of patients, and the trial results represent an estimate of the average difference in the responses of the treatment groups. Clinicians, however, engage in a process of clinical inquiry, assembling data that will allow an assessment of the appropriate choice of treatment according to more narrowly defined clinical features. We describe a method of clinical inquiry within RCTs that can enhance the applicability of results to clinical decision making. Our methods included the use of data from the Beta-Blocker Heart Attack Trial, which enrolled 3837 subjects in 31 clinical centers. The 31 centers were divided into 21 dominant centers (mortality rates higher for placebo than propranolol) and 10 divergent centers (higher mortality rates for patients randomized to propranolol). Overall, compared to placebo, propranolol reduced the risk of dying for the "average" patient from 9.8 to 7.2%. Results for patients in dominant centers (RR = 0.50) were significantly different from those in divergent centers (RR = 1.33). We identified two cotherapies--aspirin use and coronary artery surgery--that subsequently affected the benefits of propranolol in divergent centers. For patients in divergent centers, propranolol reduced the risk of dying for patients treated with aspirin and/or coronary surgery (RR = 0.39), but not for patients not receiving these therapies (RR = 1.42). We conclude that differences in results across centers of a multicenter RCT may reflect important distinctions in the clinical conditions of enrolled subjects. These distinctions help to identify subgroups of patients in which treatment that has an average overall benefit may be harmful for some patients.

Postmenopausal estrogen use and invasive versus in situ breast cancer risk.

To examine the effect of cancer histopathology on the relationship between estrogen-replacement therapy (ERT) use and breast cancer risk, we performed a case-control study of 109 postmenopausal women 45 years or older with in situ or invasive breast cancer matched to 545 controls. When in situ and invasive tumors were combined, the overall odds ratio (OR) describing the association between ERT use and breast cancer risk was not statistically significantly elevated (adjusted OR = 1.48, 95% confidence interval [CI] = 0.89-2.47). When the analyses were confined to women with invasive disease, risk estimates were uniformly higher (adjusted OR = 1.85, 95% CI = 1.00-3.45). In contrast, the overall estimate for the relationship between ERT use and in situ breast cancer was close to 1 (adjusted OR = 1.08, 95% CI = 0.42-2.77). The positive association between ERT use and invasive breast cancer we observed, and the lack of association in women with in situ disease, may represent a distinct biological difference or may be related to the small sample size of our study.

Stratified randomization for clinical trials.

Trialists argue about the usefulness of stratified randomization. For investigators designing trials and readers who use them, the argument has created uncertainty regarding the importance of stratification. In this paper, we review stratified randomization to summarize its purpose, indications, accomplishments, and alternatives. In order to identify research papers, we performed a Medline search for 1966-1997. The search yielded 33 articles that included original research on stratification or included stratification as the major focus. Additional resources included textbooks. Stratified randomization prevents imbalance between treatment groups for known factors that influence prognosis or treatment responsiveness. As a result, stratification may prevent type I error and improve power for small trials (<400 patients), but only when the stratification factors have a large effect on prognosis. Stratification has an important effect on sample size for active control equivalence trials, but not for superiority trials. Theoretical benefits include facilitation of subgroup analysis and interim analysis. The maximum desirable number of strata is unknown, but experts argue for keeping it small. Stratified randomization is important only for small trials in which treatment outcome may be affected by known clinical factors that have a large effect on prognosis, large trials when interim analyses are planned with small numbers of patients, and trials designed to show the equivalence of two therapies. Once the decision to stratify is made, investigators need to chose factors carefully and account for them in the analysis.

Clear cell adenocarcinoma of the vagina and cervix: incidence, undetected disease, and diethylstilbestrol.

We conducted an incidence study to determine the occurrence rates of clear cell adenocarcinoma (CCAC) of the vagina and cervix in young women (born in 1940 and thereafter), and a case-series analysis, focusing on the maternal history of pregnancy and delivery and in-utero exposure to diethylstilbestrol (DES). Overall, 10 cases of CCAC had been listed in the files of the Connecticut State Tumor Registry prior to the study, and each of the 10 cases were confirmed as valid. In addition, another 10 cases, all previously undetected, were found after the tissue slides of young women listed as having other cancers of the vagina and cervix were reviewed by expert pathologists, suggesting that prior estimates of the incidence rate for CCAC must be misleading unless special efforts are taken to identify undetected cases. The incidence rates of vaginal CCAC (11 cases total) were highest in 1975-1979, and decreased slightly during 1980-1982. In the cervix (nine cases total), the rate increased consistently since 1970. History of in-utero exposure to diethylstilbestrol was obtained for five of eight vaginal cases and four of eight cervical cases of CCAC. In all nine cases, exposure to diethylstilbestrol was associated with a history of bleeding during the pregnancy or prior miscarriage. We conclude that the finding of stable (or rising) incidence rates for CCAC occurring nearly 30 years after the marked decrease in diethylstilbestrol sales emphasizes the need for continued clinical and epidemiologic studies of the etiology and clinical course of CCAC.

A new index of prognostic severity for chronic asthma.

OBJECTIVE: To develop a prognostic clinical index for adults with chronic stable asthma. DESIGN: Analysis of data from a 48-week randomized, crossover trial of regular vs as-needed inhaled beta-agonist therapy. PATIENTS: Eligible patients included 70 men and women between the ages of 15 and 64 years with asthma for > 1 year. OUTCOME MEASURE: Asthma deterioration within 20 weeks, defined as either a marked decline in FEV1 (> or = 1.0 L or > or = 30% from baseline) or initiation of systemic corticosteroid therapy for asthma exacerbation. RESULTS: Three baseline factors independently predicted asthma deterioration: frequent symptoms on waking in the 4 weeks before baseline, past hospitalization for asthma, and age 35 years or older. Based on cross-stratification and consolidation of these prognostic factors, an index was developed that stratified subjects into four risk groups with distinctive deterioration rates of 9%, 21%, 39%, and 67% (p<0.001). CONCLUSION: For adults with chronic stable asthma, three simple clinical factors can be combined to stratify effectively for risk of subsequent asthma deterioration.

Randomized controlled trial in alcohol relapse prevention: role of atenolol, alcohol craving, and treatment adherence.

Previous studies suggest that beta-adrenergic blockers reduce craving levels during acute alcohol withdrawal. We conducted a new study to assess whether the daily use of atenolol by the abstinent alcoholic could maintain a blunted craving for alcohol and result in a decreased rate of relapse for alcohol abuse. The study was designed as a randomized, controlled, double-blind clinical trial. Among all 100 patients (50 atenolol, 50 placebo), only 15 stayed in the trial and remained fully abstinent for 1 year (7 atenolol, 8 placebo). Of the remaining 85 patients, 30 withdrew early while still abstinent (17 atenolol, 13 placebo). In the 57 high-risk patients who reported craving for alcohol at baseline, the treatment failure rates were 90% for patients receiving placebo, and was reduced to 65% in those who received atenolol (risk reduction = 28%, 95% confidence interval, -3% to 49%). The data from this trial also support the observation that poorer levels of treatment adherence are strongly associated with adverse outcomes for alcoholics during follow-up. This relationship was present both for patients who received atenolol and for those who received placebo.

Boosting enrollment in neurology trials with Local Identification and Outreach Networks (LIONs).

OBJECTIVE: Our purpose was to develop a geographically localized, multi-institution strategy for improving enrolment in a trial of secondary stroke prevention. METHODS: We invited 11 Connecticut hospitals to participate in a project named the Local Identification and Outreach Network (LION). Each hospital provided the names of patients with stroke or TIA, identified from electronic admission or discharge logs, to researchers at a central coordinating center. After obtaining permission from personal physicians, researchers contacted each patient to describe the study, screen for eligibility, and set up a home visit for consent. Researchers traveled throughout the state to enroll and follow participants. Outside the LION, investigators identified trial participants using conventional recruitment strategies. We compared recruitment success for the LION and other sites using data from January 1, 2005, through June 30, 2007. RESULTS: The average monthly randomization rate from the LION was 4.0 participants, compared with 0.46 at 104 other Insulin Resistance Intervention after Stroke (IRIS) sites. The LION randomized on average 1.52/1,000 beds/month, compared with 0.76/1,000 beds/month at other IRIS sites (p = 0.03). The average cost to randomize and follow one participant was $8,697 for the LION, compared with $7,198 for other sites. CONCLUSION: A geographically based network of institutions, served by a central coordinating center, randomized substantially more patients per month compared with sites outside of the network. The high enrollment rate was a result of surveillance at multiple institutions and greater productivity at each institution. Although the cost per patient was higher for the network, compared with nonnetwork sites, cost savings could result from more rapid completion of research.

The effect of acute renal failure on mortality. A cohort analysis.

OBJECTIVE: To determine if the high mortality in acute renal failure is explained by underlying illnesses (comorbidity). DESIGN: Cohort analytic study. SETTING: An 826-bed general hospital providing primary, secondary, and tertiary care. PATIENTS: From 16,248 inpatients undergoing radiocontrast procedures between 1987 and 1989, we identified 183 index subjects who developed contrast media-associated renal failure (defined as an increase in serum creatinine level of at least 25%, to at least 177 micromol/L [2 mg/dL], within 2 days of receiving contrast material) and 174 paired subjects, matched for age and baseline serum creatinine level, who underwent similar contrast procedures without developing renal failure. MAIN OUTCOME MEASURE: Death during hospitalization. RESULTS: The mortality rate in subjects without renal failure was 7%, compared with 34% in the corresponding index subjects with renal failure (odds ratio, 6.5; P<.001). After adjusting for differences in comorbidity, renal failure was associated with an odds ratio of dying of 5.5. Subjects who died after developing renal failure had complicated clinical courses characterized by sepsis, bleeding, delirium, and respiratory failure; most of these complications developed after the onset of renal failure. Deaths from renal causes were rare. CONCLUSIONS: The high mortality rate in acute renal failure is not explained by the underlying conditions alone. Renal failure appears to increase the risk of developing severe nonrenal complications that lead to death and should not be regarded as a treatable complication of serious illness.

Bladder cancer and coffee drinking: a summary of case-control research.

An association between coffee drinking and cancer of the lower urinary tract (LUT) was first suggested 20 years ago and has been the subject of many epidemiological studies. We have undertaken a critical review and statistical summary of 35 case-control studies of this association published between 1971 and 1992. Predefined methodological criteria were applied to the available reports. Studies were classified as either meeting the criteria (core studies) or failing to satisfy at least one of the requirements for design or analysis (non-core studies). The summarised data from the 8 core studies showed no evidence of an increase in risk of LUT cancer with coffee drinking in men or women after adjustment for the effects of cigarette smoking (odds ratio 1.07 [95% CI 1.00-1.14] for men, 0.91 [0.81-1.03] for women). The measures of association from the non-core studies were higher on average than those from the core studies, although the inclusion of these data in an overall summarised estimate did not substantially change the findings from the core analysis. We conclude that the best available data do not suggest a clinically important association between the regular use of coffee and development of cancer of the LUT in men or women.

The relationship of treatment adherence to the risk of death after myocardial infarction in women.

OBJECTIVE: To examine the relationship between adherence to a medical regimen and mortality following a myocardial infarction in women. DESIGN: Analysis of the female cohort entered into a randomized double-blind multicenter trial. SETTING: National Heart, Lung, and Blood Institute beta-Blocker Heart Attack Trial. PARTICIPANTS: The 602 women, aged 30 to 69 years, enrolled in the beta-Blocker Heart Attack Trial. INTERVENTION: Random assignment to propranolol hydrochloride or placebo 5 to 21 days following a myocardial infarction. MEASUREMENTS: Adherence for each patient was calculated as the mean of all quarterly adherence estimates during the course of the trial (median follow-up, 26 months). Adherence was classified as good (taking > or = 75% of medication) or poor (taking < 75% of medication). The end point was death from all causes occurring at any time during the trial, adjusted for treatment category and other clinical and sociodemographic features. RESULTS: Adherence data were available on 505 women, of whom 32 (6.3%) died. Death occurred in 13.6% of poor adherers compared with 5.6% of good adherers (relative risk, 2.4; 95% confidence interval, 1.1 to 5.6). The effect of adherence on mortality remained undiminished after adjustment for treatment category (propranolol or placebo), age, severity of myocardial infarction, congestive heart failure, smoking history, marital status, educational level, and race (adjusted relative risk of death for poor adherers, 2.5 to 3.0; P < or = .02). CONCLUSIONS: The independent effect of adherence on mortality following a myocardial infarction in women is clinically substantial, statistically significant, and similar in magnitude to that reported earlier for men.