Inaccuracy in clinical trials: effects and methods to control inaccuracy.

The increasing rate of failed trails found in mood and anxiety disorders is now being seen in Alzheimer's studies. Factors related to the administration of clinician rating scales, such as poor inter-rater reliability, poor interview quality and rater bias may be a contributing factor. Studies have found inter-rater reliability to be problematic in Alzheimer's studies, even with less subjective outcome measures. Lack of standardization of administration and scoring procedures has been identified as a major contributing factor. Remediation through better training procedures has been found to be successful, although ongoing calibration is needed to prevent rater drift. Expectancy bias and baseline score inflation is more difficult to remediate. Inflation of baseline scores increases placebo response, since lower severity has been found to be associated with higher placebo response. The use of centralized raters that are independent from study sites may help ameliorate these issues. Increased methodological research examining new approaches to these problems is warranted. The increased costs associated with this research should offset the time and expense of continuing with 'business as usual'.

Impact of generalized social anxiety disorder in managed care.

OBJECTIVE: The authors determined the costs associated with generalized social anxiety disorder in a managed care setting. METHOD: A three-phase mail and telephone survey was conducted from July to October 1998 in two outpatient clinics of a large health maintenance organization (HMO). The survey assessed direct costs, indirect costs, health-related quality of life, and clinical severity associated with generalized social anxiety disorder, both alone and with comorbid psychopathology. RESULTS: The weighted prevalence rate of current generalized social anxiety disorder was 8.2%. In the past year, only 0.5% of subjects with generalized social anxiety disorder had been accurately diagnosed. Yet 44.1% had a mental health specialty visit or had been prescribed an antidepressant, and psychiatric comorbidity was found in 43.6%. Noncomorbid generalized social anxiety disorder was associated with significantly lower health-related quality of life, work productivity, and earnings and greater utilization of health services; generalized social anxiety disorder with comorbid psychopathology was even more disabling. Suicide was attempted by 21.9% of subjects with noncomorbid generalized social anxiety disorder. Persons with average-severity generalized social anxiety disorder had probabilities of graduating from college that were 10 percentage points lower, earned wages that were 10% lower, and had probabilities of holding a technical, professional, or managerial job that were 14 percentage points lower than the comparison group. CONCLUSIONS: In a community cohort of HMO members, generalized social anxiety disorder was rarely diagnosed or treated despite being highly prevalent and associated with significant direct and indirect costs, comorbid depression, and impairment.

Mini-SPIN: A brief screening assessment for generalized social anxiety disorder.

The objectives of this study are to develop a brief self-rated screening instrument for generalized social anxiety disorder (GSAD) and to test the efficiency of the instrument. The Social Phobia Inventory (SPIN), a 17-item self-administered scale for GSAD, was given to 263 individuals with GSAD and controls. A subset of three items yielding high sensitivity and specificity for the diagnosis of GSAD was identified. This abbreviated version of the SPIN (Mini-SPIN) was administered to a group of managed care patients in conjunction with an epidemiological study of GSAD. Patients (n = 7,165) were sent a questionnaire comprising the Mini-SPIN and a brief depression screener. Respondents screening positive for GSAD on the Mini-SPIN (n = 344) were interviewed using the social phobia module of the Structured Clinical Interview for DSM-IV (SCID) to verify the diagnosis. A random sample of those who screened negative for GSAD on the Mini-SPIN were administered a similar interview to identify two control groups without GSAD for comparison (n = 673). With this information, the sensitivity, specificity, and positive and negative predictive values for the Mini-SPIN were determined (weighted for sampling). Using a cutoff score of 6 or greater, the Mini-SPIN demonstrated a sensitivity of 88.7%, specificity of 90.0%, positive predictive value of 52.5%, and negative predictive value of 98.5%. The scale possessed 90% accuracy (efficiency) in diagnosing the presence or absence of GSAD in a managed care population. The Mini-SPIN demonstrates good efficiency, supporting its utility as a screening tool for generalized social anxiety disorder.

An open-label trial of St. John's Wort (Hypericum perforatum) in obsessive-compulsive disorder.

BACKGROUND: Recent interest in and evidence for the efficacy of St. John's wort (Hypericum perforatum) for the treatment of mild-to-moderate depression has led to speculation about its efficacy in other disorders. Hypericum's mechanism of action is postulated to be via inhibition of the synaptosomal uptake of serotonin. As such, there is a suggestion that Hypericum may be effective for obsessive-compulsive disorder (OCD). METHOD: Twelve subjects were evaluated with a primary DSM-IV diagnosis of OCD of at least 12 months' duration. Treatment lasted for 12 weeks, with a fixed dose of 450 mg of 0.3% hypericin (a psychoactive compound in Hypericum) twice daily (extended-release formulation). Weekly evaluations were conducted with the Yale-Brown Obsessive Compulsive Scale (Y-BOCS), the Patient Global Impressions of Improvement Scale, and the Clinical Global Impressions of Improvement scale (CGI) and monthly evaluation with the Hamilton Rating Scale for Depression. RESULTS: A significant change from baseline to endpoint was found, with a mean Y-BOCS change of 7.4 points (p = .001). Significant change occurred at 1 week (p = .020) and continued to increase throughout the trial. At endpoint, 5 (42%) of 12 were rated "much" or "very much improved" on the clinician-rated CGI, 6 (50%) were "minimally improved," and 1 (8%) had "no change." The most common side effects reported were diarrhea (N = 3) and restless sleep (N = 2). CONCLUSION: Significant improvement was found with Hypericum, with a drop-in Y-BOCS score similar to that found in clinical trials. The fact that a significant change was found as early as 1 week into treatment suggests a possible initial placebo response, although improvement grew larger over time. Results warrant a placebo-controlled study of Hypericum in OCD.

Randomized trial of a depression management program in high utilizers of medical care.

BACKGROUND: High utilizers of nonpsychiatric health care services have disproportionally high rates of undiagnosed or undertreated depression. OBJECTIVE: To determine the impact of offering a systematic primary care-based depression treatment program to depressed "high utilizers" not in active treatment. DESIGN: Randomized clinical trial. SETTING: One hundred sixty-three primary care practices in 3 health maintenance organizations located in different geographic regions of the United States. PATIENTS: A group of 1465 health maintenance organization members were identified as depressed high utilizers using a 2-stage telephone screening process. Eligibility criteria were met by 410 patients and 407 agreed to enroll: 218 in the depression management program (DMP) practices and 189 in the usual care (UC) group. INTERVENTION: The DMP included patient education materials, physician education programs, telephone-based treatment coordination, and antidepressant pharmacotherapy initiated and managed by patients' primary care physicians. MAIN OUTCOME MEASURES: Depression severity was measured using the Hamilton Depression Rating Scale (Ham-D) and functional status using the Medical Outcomes Study 20-item short form (SF-20) subscales. Outpatient visit and hospitalization rates were measured using the health plan's encounter data. RESULTS: Based on an intent-to-treat analysis, at least 3 antidepressant prescriptions were filled in the first 6 months by 151 (69.3%) of 218 of DMP patients vs 35 (18.5%) of 189 in UC (P < .001). Improvements in Ham-D scores were significantly greater in the intervention group at 6 weeks (P = .04), 3 months (P = .02), 6 months (P < .001), and 12 months (P < .001). At 12 months, DMP intervention patients were more improved than UC patients on the mental health, social functioning, and general health perceptions scales of the SF-20 (P < .05 for all). CONCLUSION: In depressed high utilizers not already in active treatment, a systematic primary care-based treatment program can substantially increase adequate antidepressant treatment, decrease depression severity, and improve general health status compared with usual care.

Computerized assessment of depression and anxiety over the telephone using interactive voice response.

We examined the reliability and validity of computer-administered versions of the Hamilton Depression (HAMD) and Hamilton Anxiety (HAMA) Rating Scales that were administered over the telephone using Interactive Voice Response (IVR). In two identical studies (HAMD: N = 113, HAMA: N = 74), both the IVR- and clinician-administered versions were administered in a counterbalanced order to a heterogeneous sample of subjects with psychiatric disorders and controls. Both the IVR HAMD and HAMA demonstrated adequate internal-consistency reliability (.90 and .93, respectively) and test-retest reliability (.74 and .97, respectively). The correlation between the IVR and clinician was high (HAMD = .96; HAMA = .65). The mean score difference between the IVR and clinician versions was less than one point for both the HAMD (.69 of a point) and HAMA (.60 of a point). It took subjects 12.23 minutes to complete the IVR HAMD, compared to 15.21 minutes for the clinician version; and 11.27 minutes for the IVR HAMA, compared to 15.33 minutes for the clinician (p < .001 for both comparisons). Subjects rated the clinician better in the areas of how much they liked being interviewed and how well they were able to describe their feelings. However, they were significantly more embarrassed with the clinician than with the IVR. Results support the psychometric properties of the IVR versions of the HAMD and HAMA scales. IVR technology presents new opportunities for expanding the utility of computerized clinical assessment.

Computer-administered rating scales for social anxiety in a clinical drug trial.

Computer-administered versions of two clinician-administered symptom rating scales for social anxiety (the Liebowitz Social Anxiety Scale [LSAS] and the Brief Social Phobia Scale [BSPS]) and one paper-and-pencil scale (the Fear Questionnaire) were developed and utilized in a clinical trial for social phobia. The reliability and validity of the computer versions were examined, as were their equivalence to the traditional versions. Correlations between the computer and original versions were high at baseline, and remained high throughout the study. The internal consistency reliability of the computer scales was also high, and almost identical to the original versions. Mean score differences between computer and original versions were not significant at baseline, and no significant differences were found between computer and traditional versions on the amount of change detected from baseline to endpoint. Seventy-seven percent of subjects felt that the computer did not interfere with their visit at baseline and a plurality (36%) preferred the computer, with 30% preferring the clinician and 34% having no preference. By the end of the study, the plurality (41%) had no preference, with 27% preferring the computer and 32% preferring the clinician. Results support the use of these computer-administered symptom rating scales of social anxiety as a viable alternative to the clinician-administered versions with this subset of patients, which should offer researchers and clinicians a reliable and cost-effective method for evaluating social phobia.

Self-treatment for obsessive compulsive disorder using a manual and a computerized telephone interview: a U.S.-U.K. study.

Bt steps is a patient-centered behavioral therapy program that uses a manual and a computer-driven interactive voice response system to assess and treat obsessive compulsive disorder. This nine-step program contains a self-assessment module and a self-treatment module that provides teaching on exposure and ritual prevention. The patient reads about the steps in a manual and then uses a touch-tone telephone to contact the program, in which a recorded voice conducts the interview. Of 40 patients in an open 12-week trial in the United States and London, 35 completed the self-assessment module, and 17 completed at least two sessions of exposure and ritual prevention. The system produced statistically significant improvements on measures of obsessive compulsive disorder.

Behavioral versus pharmacological treatments of obsessive compulsive disorder: a meta-analysis.

The goal of the study was to provide a quantitative analysis of the relative efficacy of all five currently available serotonin reuptake inhibitors (SRIs) and behavior therapy [exposure and response prevention (ERP)] for obsessive compulsive disorder. The relationship between effect size and methodological characteristics was also empirically examined. A search was conducted of several computerized databases covering the dates from 1973 to 1997. Seventy-seven studies were identified, yielding 106 treatment comparisons involving 4641 patients. Effect sizes were analyzed between individual interventions and between intervention class [SRI, ERP or the combined treatment of an SRI with ERP(ERP/SRI)]. Data were analyzed both before and after controlling for methodological variables. The effect size for clomipramine (CMI) was significantly greater than the other SRIs, with the exception of fluoxetine (FLX). CMI was not significantly greater than ERP or ERP/SRI. As a class, ERP was significantly greater than SRIs as a whole. Effect sizes were larger for studies without a control group or random assignment, for self-reported outcome measures, and varied significantly by method of effect size calculation. Year of publication was significantly related to effect size. When controlling for these methodological variables, CMI was not significantly greater than FLX or fluvoxamine (FLV), and ERP was no longer significantly greater than the SRIs as a whole. No significant difference was found between CMI and the other SRIs as a group in head to head trials. No differences in drop-out rates were found. CMI stands out from the other SRIs. This difference is probably not clinically significant enough to warrant first choice treatment, given CMI's greater lethality in overdose. The choice between an SRI or ERP is dominated primarily by the infrequent availability of ERP and to a lesser degree by personal preference. Methodological differences significantly impact effect size.

Administration of the Hamilton Depression Rating Scale using interactive voice response technology.

We compared the reliability and equivalence of scores on the Hamilton rating scale for depression obtained by an interactive voice response (IVR) system and by human raters among 367 subjects 18 to 79 years old. Structured clinical interviews indicated that 93 of these subjects met the DSM-IV criteria for current major depression. We also examined short forms using the IVR data. The results obtained with the long (17-item) IVR interview correlated with those of the face-to-face interview, as did the results of the short (six- and eight-item) IVR interviews. In a random sample of 264 subjects, linear transformations of IVR results produced scores equivalent to clinical scores in the validation sample. We conclude that computerized assessment of the severity of depression by touch-tone telephone presents expanded opportunities for outcome research.

A computer-administered telephone interview to identify mental disorders.

CONTEXT: Common mental disorders are often not identified in primary care settings. OBJECTIVE: To evaluate the validity and clinical utility of a telephone-assisted computer-administered version of Primary Care Evaluation of Mental Disorders (PRIME-MD), a brief questionnaire and interview instrument designed to identify psychiatric disorders in primary care patients. DESIGN: Comparison of diagnoses obtained by computer over the telephone using interactive voice response (IVR) technology vs those obtained by a trained clinician over the telephone using the Structured Clinical Interview for DSM-IV [Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition] Diagnosis (SCID). A subsample also received the clinician-administered version of PRIME-MD. PATIENTS: Outpatients (N=200) from 4 primary care clinics, an eating disorders clinic, an alcohol treatment facility, psychiatric outpatients, and community controls. SETTING: Interviews conducted by telephone, except for face-to-face administration of PRIME-MD, which was conducted at either the primary care clinic or a research clinic. MEASUREMENTS AND MAIN RESULTS: Prevalence rates for any psychiatric disorder were similar between diagnoses made by the computer and those made by a mental health professional using the SCID (60.0% vs 58.5%). Prevalence rates for individual diagnoses were generally similar across versions. However, primary care patients reported twice as much alcohol abuse on the computer (15.0%) as on either the SCID (7.5%) or the clinician-administered PRIME-MD (7.5%). Using the SCID as the criterion, both the computer- and clinician-administered versions of PRIME-MD demonstrated high and roughly equivalent levels of sensitivity and specificity. Overall agreement (K) for any diagnosis was 0.67 for the computer-administered PRIME-MD and 0.70 for the clinician-administered PRIME-MD. CONCLUSIONS: The computer-administered PRIME-MD appears to be a valid instrument for assessing psychopathology in primary care patients. Interactive voice response technology allows for increased availability, and provides primary care physicians with information that will increase the quality of patient care without additional physician time and at minimal expense.

Brofaromine for social phobia: a multicenter, placebo-controlled, double-blind study.

The safety and efficacy of brofaromine, a reversible and selective monoamine oxidase inhibitor, were examined in a multicenter trial of 102 outpatients with social phobia. After a 1-week placebo washout, subjects were randomly assigned to 10 weeks of treatment with either brofaromine (N = 52) or placebo (N = 50). Brofaromine dosage began at 50 mg/day and was titrated to a maximum of 150 mg/day, depending on treatment response. Brofaromine produced a significantly greater change from baseline in Liebowitz Social Anxiety Scale (LSAS) scores compared with placebo, F(1) = 6.01, p < 0.016. Mean LSAS scores decreased from 81.8 at baseline to 62.6 at endpoint for brofaromine, t = 5.41,p < 0.001, and from 79.8 to 70.7 for placebo, t = 3.66, p < 0.001. Eleven of the 14 brofaromine early terminators discontinued because of adverse experiences, as did 4 of the 17 placebo early terminators. Side effects more common with brofaromine than placebo included insomnia, dizziness, dry mouth, anorexia, tinnitus, and tremor. No clinically significant variations in vital signs or laboratory values were found. The findings are consistent with the clinical efficacy for the treatment of social phobia.

Computerized screening for psychiatric disorders in an outpatient community mental health clinic.

OBJECTIVE: This study examined the validity and utility of two types of computer-administered versions of a screening interview, PRIME-MD (Primary Care Evaluation of Mental Disorders), in a mental health setting: one administered by desktop computer and one by computer using a touch-tone telephone and interactive voice response (IVR) technology. METHODS: Fifty-one outpatients at a community mental health clinic were given both IVR and desktop PRIME-MD and the Structured Clinical Interview for DSM-IV (SCID-IV), which was administered by a clinician, in a counterbalanced order. Diagnoses were also obtained from charts. RESULTS: Prevalence rates found by both computer interviews were similar to those obtained by the SCID-IV for the presence of any diagnosis, any affective disorder, and any anxiety disorder. Prevalence rates for specific diagnoses were also similar to those found by the SCID-IV except for dysthymia, obsessive-compulsive disorder, and panic disorder; the first two conditions were found to be more prevalent by the computer, and panic disorder was more prevalent by the SCID. Compared with the prevalence rates in the charts, the rates found by the computer were higher for anxiety disorders, particularly for obsessive-compulsive disorder and social phobia. Using the SCID-IV as the criterion, both computer-administered versions of PRIME-MD had high sensitivity, specificity, and positive predictive value for most diagnoses. No significant difference was found in how well patients liked each form of interview. CONCLUSIONS: Results support the validity and utility of both desktop and IVR PRIME-MD for gathering information from mental health patients about certain diagnoses.

Effect of primary care treatment of depression on service use by patients with high medical expenditures.

OBJECTIVE: The study examined the impact of identifying and treating depression among patients who had a history of high medical expenditures. Effects on service use, disability, and quality of life were measured. METHODS: A total of 786 high users of services from two primary care clinics and an equal number of randomly selected patients who were not high users were screened for depression using the Medical Outcomes Study (MOS) depression screen. High-user patients who screened positive were subsequently seen by their primary care physician for a diagnostic interview. The 20 patients with a confirmed diagnosis of depression were offered open-label antidepressant treatment by their primary care physician for six months. RESULTS: All 20 patients completed the six-month study. Treatment resulted in significant reductions in depression and significant improvements in quality of life in the areas of social functioning, general health, mental health, physical functioning, emotional role functioning, and vitality. Days of missed work per month were reduced, and the percentage of patients who reported not being at all impaired by their depression at work increased. Costs for service use fell from $13.28 to $6.75 per day; when costs for the treatment study were added, the daily service use cost was $12.55. CONCLUSIONS: Identification and treatment of depression among patients with a history of high medical expenditures improved depression and increased work productivity and quality of life. Service use decreased with treatment. A larger sample and control group are needed to determine if treatment is associated with a statistically significant decrease in medical expenditures.

Computer-administered clinical rating scales. A review.

While clinician-administered symptom rating scales are the most commonly used outcome measures in pharmaceutical research, error variance due to poor inter-rater reliability increases the risk of type II errors in multi-center clinical trials. Such error variance could obscure true differences between active drug and placebo, or between two comparator compounds. Computer-administered versions of symptom rating scales originally designed to be administered by trained clinicians offer a solution to this problem. This paper reviews the empirical data on the reliability, validity and equivalence of computer-administered rating scales. Computer-administered versions of clinician-administered scales are now available for the assessment of depression, anxiety, obsessive-compulsive disorder, and social phobia. Validation studies support the reliability, validity and equivalence of these scales. Patient reaction has been positive, with patients generally more honest with and often preferring the computer for assessing sensitive areas such as suicide, alcohol or drug abuse, sexual behavior, or HIV related symptoms. Applications using Interactive Voice Response (IVR) technology facilitate longitudinal monitoring of patients without requiring office visits to collect data, increase the accessibility of information to the clinician, and the quality of patient care through more informed decision making. When used in accordance with established ethical guidelines, computers offer a reliable, inexpensive, accessible, and time-efficient means of assessing psychiatric symptoms.

Medical costs attributed to depression among patients with a history of high medical expenses in a health maintenance organization.

BACKGROUND: While previous studies have compared medical utilization between depressed and nondepressed patients, we conducted a study that focused specifically on patients who had a history of high medical expenditures. METHODS: This study was designed to determine whether a positive screen for depression is predictive of continued high medical expenditures. Medical utilization data were obtained on 50,000 patients enrolled in the DeanCare health maintenance organization for 2 consecutive years. Consistent high utilizers were identified based on the medical utilization costs (paid by the health maintenance organization) for those 2 consecutive years, 1992 and 1993. A depression screen based on the Medical Outcomes Survey was mailed to 786 high utilizers. Their costs were determined for 1994. Regression analyses identified 1994 costs associated with depression, adjusting for age, sex, benefits package, and medical comorbidity. RESULTS: Depressed high utilizers were more likely than nondepressed high utilizers to have higher medical costs in 1994. Among high utilizers, depressed patients' 1994 costs were significantly higher ($5764 vs $4227; P < .001), although expenditures for depressed and nondepressed high utilizers were similar for the previous 2 years. The total medical cost associated with depression in 1994, adjusted for age, sex, benefits package, and medical comorbidity, was $1498 per patient. CONCLUSIONS: In the third year (1994), a positive Medical Outcomes Survey screen for depression in high utilizers was associated with $1498 in higher medical costs. The average actual amount spent on depression treatment accounted for only a small portion of total medical costs for depressed high utilizers in the third year.

Decision support for patient care: computerized rating scales.

Cost-effective patient assessment instruments continue to be developed and improved. Computer technology advances enable these instruments to be administered more efficiently, with more rapid feedback to clinical decision makers and patients. This article describes several of these instruments and how they are used in primary care and behavioral healthcare settings for treatment planning and disease management. The authors are eminent research pioneers and leaders in this field.