Pilot Study of an Online Parent-Training Course for Disruptive Behavior with Live Remote Coaching for Practitioners.

OBJECTIVE: Many clinicians find it challenging to obtain training in evidence-based interventions, including behavioral parent training, which is considered the front-line treatment for children with disruptive behaviors (Chacko et al., 2017). Workshops, ongoing consultation, and feedback provided in person are effective, yet are rarely feasible for clinicians in the field (Fixsen, Blase, Duda, Naoom, & Van Dyke, 2010). The purpose of the present study was to conduct a preliminary assessment of an online tutorial combined with live remote coaching for training mental health professionals in behavioral parent training. METHOD: Participants in this pretest-posttest open trial were 22 clinicians and graduate students (73% female) from around the United States. RESULTS: The web platform operated successfully, and clinicians found the training to be highly satisfactory. Compared to pre-training, participants demonstrated large improvements in knowledge about disruptive behavior and behavioral parent training and performed significantly better on demonstrations of skill in administering behavioral parent-training components. CONCLUSIONS: An online course combined with live remote coaching is a promising methodology for significantly increasing the number of clinicians trained in evidence-based interventions for disruptive behavior in children. Next steps for evaluation and expansion of this training model are discussed.

Web-Based Therapist Training in Interpersonal Psychotherapy for Depression: Pilot Study.

BACKGROUND: Training mental health professionals to deliver evidence-based therapy (EBT) is now required by most academic accreditation bodies, and evaluating the effectiveness of such training is imperative. However, shortages of time, money, and trained EBT clinician teachers make these challenges daunting. New technologies may help. The authors have developed the first empirically evaluated comprehensive Internet therapist training program for interpersonal psychotherapy (IPT). OBJECTIVE: The aim of this study was to examine whether (1) the training protocol would increase clinicians' knowledge of IPT concepts and skills and (2) clinicians would deem the training feasible as measured by satisfaction and utility ratings. METHODS: A total of 26 clinicians enrolled in the training, consisting of (1) a Web-based tutorial on IPT concepts and techniques; (2) live remote training via videoconference, with trainees practicing IPT techniques in a role-play using a case vignette; and (3) a Web-based portal for therapists posttraining use to help facilitate implementation of IPT and maintain adherence over time. RESULTS: Trainees' knowledge of IPT concepts and skills improved significantly (P<.001). The standardized effect size for the change was large: d=2.53, 95% CI 2.23-2.92. Users found the technical features easy to use, the content useful for helping them treat depressed clients, and felt the applied training component enhanced their professional expertise. Mean rating of applied learning was 3.9 (scale range from 1=very little to 5=a great deal). Overall satisfaction rating was 3.5 (range from 1=very dissatisfied to 4=very satisfied). CONCLUSIONS: Results support the efficacy and feasibility of this technology in training clinicians in EBTs and warrant further empirical evaluation.

Erratum to: Integrating technology into cognitive behavior therapy for adolescent depression: a pilot study.

[This corrects the article DOI: 10.1186/s12991-015-0077-8.].

Comparison of Site-Based Versus Central Ratings in a Study of Generalized Anxiety Disorder.

Lack of standardization across sites and raters, poor interrater reliability, and possible scoring bias affecting the primary outcome measure contribute to a high failure rate in anxiety trials. Remote centralized raters who are blinded to protocol inclusion and exclusion criteria as well as visit number may standardize assessments across raters and eliminate scoring bias, decreasing placebo response and thereby increasing signal detection. The purpose of the primary study was to test the safety and efficacy of an anxiolytic in a double-blind, placebo-controlled (no active comparator), multicenter trial. However, there was an additional prospective objective to explore site ratings compared with remote centralized ratings in the cohort of subjects on placebo. Site raters assessed subjects 6 times over an 8-week period. The primary outcome measure was the week 8 site-rated Hamilton Anxiety Scale (HAM-A). Remote centralized raters by telephone independently rated these subjects on the HAM-A at baseline and week 6. Of the 122 subjects selected by site raters and therefore randomized, remote centralized raters would have admitted 59 (48%) and excluded 63 (52%), based on their HAM-A ratings. The mean change from baseline in HAM-A total score in the placebo group admitted to the study by site raters was 9.3, significantly higher than the 5.9 point mean change on placebo as measured by the remote centralized raters.The data are consistent with the potential for qualification bias at baseline when rated by sites. The results make a strong case for using strategies to ensure that baseline scoring is truly independent of the pressure to enroll.

Integrating technology into cognitive behavior therapy for adolescent depression: a pilot study.

BACKGROUND: Rapid advances in information technology and telecommunications have resulted in a dramatic increase in the use of mobile devices and the internet to enhance and facilitate access to treatment. Cognitive behavior therapy (CBT) is an empirically based treatment that is well suited for enhancement by new technologies, particularly with youth. To facilitate the dissemination of this evidence-based treatment, we developed a technology-enhanced CBT intervention for the treatment of adolescent depression consisting of (1) online therapist training (2) in-session use of tablets for teaching clients CBT concepts and skills, and (3) text messaging for between session homework reminders and self-monitoring. METHODS: Eighteen licensed clinicians (social workers n = 7, psychologists n = 9) were randomized to have their patients receive either the intervention (CBT) or treatment as usual (TAU). Each clinician treated four adolescents for 12 weeks. Clinicians in the CBT arm completed an online tutorial on CBT treatment of adolescent depression, then received an iPad with access to patient education materials for teaching CBT concepts to patients during sessions. Individualized text messages were integrated into treatment for homework reminders, support, and outcomes measurement. Outcome measures included a 49-item multiple choice test for tutorial effectiveness; the system usability scale (SUS) for user satisfaction; quick inventory of depressive symptomatology-adolescent version (QIDS-A-Pat); and clinician and patient ratings on the therapeutic alliance scale for adolescents (TASA). RESULTS: A significant increase in knowledge of CBT concepts was found after completing the tutorial, t(8) = 7.02, p < 0.001. Clinician and patient ratings of user satisfaction were high for both the iPad teaching tools, and the text messaging. Ninety-five percent of teens said reviewing their text messages with their therapist was helpful, and all said they would use text messaging in treatment again. Ratings of the therapeutic alliance were higher in the CBT arm t(131) = 4.03, p = 0.001. A significant reduction in depression was found in both groups [t(34) = 8.453, p < 0.001 and t(29) = 6.67, p < 0.001 for CBT and TAU, respectively). Clinical ratings of improvement were greater on all outcome measures for the CBT arm; however, none reached statistical significance. Effect sizes (Cohen's d) ranged from small (QIDS-A) to large (TASA). CONCLUSIONS: Results support the feasibility of this technology-enhanced CBT intervention as a means of improving CBT treatment of adolescent depression and may help address the critical shortage of therapists trained on empirically based treatments.

Computer-assisted cognitive behavior therapy for obsessive-compulsive disorder: a randomized trial on the impact of lay vs. professional coaching.

BACKGROUND: The purpose of the study was to examine the impact of computerized cognitive behavior therapy (CBT) self-help treatment for obsessive-compulsive disorder (OCD) (BT Steps) both alone and when supported by coaching from either a lay non-therapist coach or an experienced CBT therapist. METHODS: Eighty-seven subjects with clinically significant OCD were recruited through newspaper ads and randomly assigned to receive 12 weeks of treatment with either BT Steps alone (n = 28), BT Steps with non-therapist coaching (n = 28), or BT Steps with CBT therapist coaching (n = 31). Subjects worked on BT Steps at their own pace. Subjects receiving BT Steps alone received a welcome call from the project manager. Subjects randomized to either of the coaching arms received regularly scheduled weekly phone calls for coaching, encouragement, and support. No formal therapy was provided by the coaches; thus, both lay and CBT coaches completed the same tasks. RESULTS: All three treatment arms showed a significant reduction in Yale-Brown Obsessive Compulsive Scale (YBOCS) scores, with mean (SD) changes of 6.5 (5.7), 7.1 (6.1), and 6.5 (6.1) for the no coaching, lay coaching, and therapist coaching arms, respectively (all p's < .001). These represent effect sizes of 1.16, 1.41, and 1.12, respectively. No significant differences were found between treatment arms on YBOCS change scores, F(2) = 0.10, p = .904, or number of exposures sessions done (F(2) = 0.033, p = .967). When asked which method of therapy (computer vs. clinician) they preferred, 48% said computer, 33% said face-to-face therapy, and 19% had no preference. CONCLUSIONS: Results support the use of online self-help for the treatment of moderate OCD. The addition of coaching by either a lay coach or a CBT therapist coach did not significantly improve outcomes.

An evaluation of treatment response and remission definitions in adult obsessive-compulsive disorder: A systematic review and individual-patient data meta-analysis.

INTRODUCTION: Expert consensus operationalized treatment response and remission in obsessive-compulsive disorder (OCD) as a Yale-Brown Obsessive-Compulsive Scale (Y-BOCS) reduction ≥35% and score ≤12 with ≤2 on Clinical Global Impressions Improvement (CGI-I) and Severity (CGI-S) scales, respectively. However, there has been scant empirical evidence supporting these definitions. METHODS: We conducted a systematic review and an individual participant data meta-analysis of randomized-controlled trials (RCTs) in adults with OCD to determine optimal Y-BOCS thresholds for response and remission. We estimated pooled sensitivity/specificity for each percent reduction threshold (response) or posttreatment score (remission) to determine response and remission defined by a CGI-I and CGI-S ≤ 2, respectively. RESULTS: Individual participant data from 25 of 94 eligible RCTs (1235 participants) were included. The optimal threshold for response was ≥30% Y-BOCS reduction and for remission was ≤15 posttreatment Y-BOCS. However, differences in sensitivity and specificity between the optimal and nearby thresholds for response and remission were small with some uncertainty demonstrated by the confidence ellipses. CONCLUSION: While the empirically derived Y-BOCS thresholds in our meta-analysis differ from expert consensus, given the predominance of data from more recent trials of OCD, which involved more refractory participants and novel treatment modalities as opposed to first-line therapies, we recommend the continued use of the consensus definitions.

A Web-Based Therapist Training Tutorial on Prolonged Grief Disorder Therapy: Pre-Post Assessment Study.

BACKGROUND: Prolonged grief disorder (PGD) is a newly recognized mental disorder characterized by pervasive intense grief that persists longer than cultural or social expectations and interferes with functioning. The COVID-19 epidemic has resulted in increased rates of PGD, and few clinicians feel confident in treating this condition. PGD therapy (PGDT) is a simple, short-term, and evidence-based treatment developed in tandem with the validation of the PGD diagnosis. To facilitate the dissemination of PGDT training, we developed a web-based therapist tutorial that includes didactic training on PGDT concepts and principles as well as web-based multimedia patient scenarios and examples of clinical implementation of PGDT. OBJECTIVE: We aimed to evaluate user satisfaction with the tutorial and whether the tutorial increased trainees' knowledge of PGDT principles and procedures. Moreover, we included a small number of pilot questions to evaluate the PGDT-related clinical skills. METHODS: This study evaluated tutorial learning using a pre- and poststudy design. Participants were recruited from professional organization mailing lists, announcements to graduates of the Columbia School of Social Work, and through word of mouth. After signing consent, participants completed a brief demographic survey, a 55-item multiple-choice prestudy test on the concepts and principles of PGD and PGDT covered in the tutorial, and a 4-item pilot web-based prestudy test to gauge PGD clinical implementation skills. The link to the course content was then activated, and participants were given 8 weeks to complete the 11-module tutorial containing information, web-based exercises, simulated patient and video examples, and self-tests. RESULTS: Overall, 406 clinicians signed consent, and 236 (58.1%) started the tutorial. Of these, 83.1% (196/236) completed all 11 modules. Trainee scores on our PDGT assessment improved substantially from pretraining to the postmodule assessment, with the total number of correct answers increasing from a mean of 29 (SD 5.5; 52.7% correct) to 36.7 (SD 5.2; 66.7% correct; t195=18.93; P<.001). In addition, the trainee's implementation scores on 4 clinical vignettes increased from 2.6 (SD 0.7) correct out of 4 to 3.1 (SD 0.4) out of 4 (t188=7.02; P<.001). Effect sizes (Cohen d) were 1.44 (95% CI 1.23-1.65) for PDGT assessment and 1.06 (95% CI 0.84-1.29) for implementation. Trainees found the tutorial interesting, enjoyable, clearly presented, and useful for professional development. They endorsed a mean score of 3.7 (SD 0.47) on a 1 to 4 scale of agreement with recommending the course to others and feeling satisfied with the tutorial, and a mean of 3.3 (SD 0.57) with feeling able to apply the skills with clients. CONCLUSIONS: This pilot study provides support for the usefulness of this web-based training for teaching clinicians how to administer PGDT. The addition of patient scenarios for clinical implementation strategies holds promise for increasing the effectiveness of PGDT training and other evidence-based treatments. TRIAL REGISTRATION: ClinicalTrials.gov NCT05121792; https://www.clinicaltrials.gov/ct2/show/NCT05121792.

The use of videoconferencing with patients with psychosis: a review of the literature.

Videoconferencing has become an increasingly viable tool in psychiatry, with a growing body of literature on its use with a range of patient populations. A number of factors make it particularly well suited for patients with psychosis. For example, patients living in remote or underserved areas can be seen by a specialist without need for travel. However, the hallmark symptoms of psychotic disorders might lead one to question the feasibility of videoconferencing with these patients. For example, does videoconferencing exacerbate delusions, such as paranoia or delusions of reference? Are acutely psychotic patients willing to be interviewed remotely by videoconferencing? To address these and other issues, we conducted an extensive review of Medline, PsychINFO, and the Telemedicine Information Exchange databases for literature on videoconferencing and psychosis. Findings generally indicated that assessment and treatment via videoconferencing is equivalent to in person and is tolerated and well accepted. There is little evidence that patients with psychosis have difficulty with videoconferencing or experience any exacerbation of symptoms; in fact, there is some evidence to suggest that the distance afforded can be a positive factor. The results of two large clinical trials support the reliability and effectiveness of centralized remote assessment of patients with schizophrenia.

Web-based training in early autism screening: results from a pilot study.

BACKGROUND: Lack of familiarity with early signs of autism by community service providers has resulted in significant delays in children receiving early intervention services necessary to improve long-term outcomes. The Screening Tool for Autism in Toddlers and Young Children (STAT) was specifically developed to identify early behavioral features of autism. Although STAT training has been available for years, access is limited because of few STAT trainers and geographic concerns. This study evaluated the efficacy and acceptability of Web-based training of the STAT as a means of increasing accessibility to this training. MATERIALS AND METHODS: Thirty professionals from three geographic areas participated. Roughly 1 of 3 had little or no training on autism assessment. The tutorial contains a general overview, administration and scoring conventions, and item-specific content and concepts. Participants completed a pretest and then completed the STAT tutorial at their own pace, followed by a post-test and a user satisfaction questionnaire. RESULTS: Mean scores on STAT concepts significantly improved after taking the tutorial (p<0.001). At pretest, only 1 person (3%) obtained correct scores on at least 80% of the items (a priori cutoff for a "pass"), compared with 22 (73%) at post-test (p<0.001). The majority of trainees enjoyed taking the tutorial, thought it was well organized, relevant, interesting, and useful, and felt it was easy to understand and operate. DISCUSSION: Results support Web-based training as a promising method for promoting early identification of autism and may help overcome problems associated with the critical shortage of autism-screening professionals.

A web-based tutorial for parents of young children with autism: results from a pilot study.

OBJECTIVE: Early intervention can significantly improve long-term outcomes for children with autism. Unfortunately, many children do not receive early intervention services due to a critical shortage of trained professionals in this area. To bridge this gap, we evaluated a Web-based parent training tutorial (Enhancing Interactions), based on evidence-based practices and utilizing the Web-based platform to maximize learning. METHODS: Twenty-three parents with a child between 18 months and 6 years with an autism spectrum disorder participated. Pre- and posttest scores of parents' knowledge were used to evaluate tutorial effectiveness. The system usability scale (SUS) evaluated technical user-friendliness and the user satisfaction questionnaire (USQ), gauged satisfaction with content. RESULTS: The mean number of correct items on the posttest significantly increased, from 12.6 to 20.4, p<0.001. The mean SUS score was 85 (standard deviation=17), corresponding to a score of "excellent." All participants found the tutorial user friendly, well integrated, and 96% (all but one participant) thought it was easy to use, felt confident using the technical features, and would use a tutorial like this again. On the USQ, all participants found that the tutorial was well organized, clearly presented, and easy to understand; that it increased their knowledge about communicating with their child; and that they felt capable of applying these techniques with their child. CONCLUSIONS: The tutorial appears effective in increasing parents' knowledge with high user satisfaction.

Site versus centralized raters in a clinical depression trial: impact on patient selection and placebo response.

The use of centralized raters who are remotely linked to sites and interview patients via videoconferencing or teleconferencing has been suggested as a way to improve interrater reliability and interview quality. This study compared the effect of site-based and centralized ratings on patient selection and placebo response in subjects with major depressive disorder. Subjects in a 2-center placebo and active comparator controlled depression trial were interviewed twice at each of 3 time points: baseline, 1-week postbaseline, and end point--once by the site rater and once remotely via videoconference by a centralized rater. Raters were blind to each others' scores. A site-based score of greater than 17 on the 17-item Hamilton Depression Rating Scale (HDRS-17) was required for study entry. When examining all subjects entering the study, site-based raters' HDRS-17 scores were significantly higher than centralized raters' at baseline and postbaseline but not at end point. At baseline, 35% of subjects given an HDRS-17 total score of greater than 17 by a site rater were given an HDRS total score of lower than 17 by a centralized rater and would have been ineligible to enter the study if the centralized rater's score was used to determine study entry. The mean placebo change for site raters (7.52) was significantly greater than the mean placebo change for centralized raters (3.18, P < 0.001). Twenty-eight percent were placebo responders (>50% reduction in HDRS) based on site ratings versus 14% for central ratings (P < 0.001). When examining data only from those subjects whom site and centralized raters agreed were eligible for the study, there was no significant difference in the HDRS-17 scores. Findings suggest that the use of centralized raters could significantly change the study sample in a major depressive disorder trial and lead to significantly less change in mood ratings among those randomized to placebo.

Computer-Assisted Psychiatric Diagnosis.

Accurate psychiatric diagnosis is critical for both sound clinical interventions and valid research methodology. Over the years, attempts to improve diagnostic reliability and accuracy led to the development of more explicit operationalized diagnostic criteria, starting with DSM-III, and subsequently fully structured and semistructured diagnostic interviews.1 As diagnostic assessment changed and with advances in technology, the use of computers soon developed in parallel to improve the reliability and validity of psychiatric diagnosis. As far back as 1968, computers were used to help clinicians formulate psychiatric diagnoses, by helping them process clinical information according to diagnostic algorithms.2 Since that time, there has been an exponential rise in the use of technology in clinical research and practice. Indeed, computers have been used both to transition diagnostic interviews from paper-and-pencil format to instruments that are clinician-administered via an electronic platform and to create self-report versions of clinician-administered diagnostic interviews. We will discuss each of these in turn.

Development of Three Web-Based Computerized Versions of the Kiddie Schedule for Affective Disorders and Schizophrenia Child Psychiatric Diagnostic Interview: Preliminary Validity Data.

OBJECTIVE: To present initial validity data on three web-based computerized versions of the Kiddie Schedule for Affective Disorders and Schizophrenia (KSADS-COMP). METHOD: The sample for evaluating the validity of the clinician-administered KSADS-COMP included 511 youths 6-18 years of age who were participants in the Child Mind Institute Healthy Brain Network. The sample for evaluating the parent and youth self-administered versions of the KSADS-COMP included 158 youths 11-17 years of age recruited from three academic institutions. RESULTS: Average administration time for completing the combined parent and youth clinician-administered KSADS-COMP was less time than previously reported for completing the paper-and-pencil K-SADS with only one informant (91.9 ± 50.1 minutes). Average administration times for the youth and parent self-administered KSADS-COMP were 50.9 ± 28.0 minutes and 63.2 ± 38.3 minutes, respectively, and youths and parents rated their experience using the web-based self-administered KSADS-COMP versions very positively. Diagnoses generated with all three KSADS-COMP versions demonstrated good convergent validity against established clinical rating scales and dimensional diagnostic-specific ratings derived from the KSADS-COMP. When parent and youth self-administered KSADS-COMP data were integrated, good to excellent concordance was also achieved between diagnoses derived using the self-administered and clinician-administered KSADS-COMP versions (area under the curve = 0.89-1.00). CONCLUSION: The three versions of the KSADS-COMP demonstrate promising psychometric properties, while offering efficiency in administration and scoring. The clinician-administered KSADS-COMP shows utility not only for research, but also for implementation in clinical practice, with self-report preinterview ratings that streamline administration. The self-administered KSADS-COMP versions have numerous potential research and clinical applications, including in large-scale epidemiological studies, in schools, in emergency departments, and in telehealth to address the critical shortage of child and adolescent mental health specialists. CLINICAL TRIAL REGISTRATION INFORMATION: Computerized Screening for Comorbidity in Adolescents With Substance or Psychiatric Disorders; https://clinicaltrials.gov/; NCT01866956.

THE DEPRESSION INVENTORY DEVELOPMENT SCALE: Assessment of Psychometric Properties Using Classical and Modern Measurement Theory in a CAN-BIND Trial.

Objective: The goal of the Depression Inventory Development (DID) project is to develop a comprehensive and psychometrically sound rating scale for major depressive disorder (MDD) that reflects current diagnostic criteria and conceptualizations of depression. We report here the evaluation of the current DID item bank using Classical Test Theory (CTT), Item Response Theory (IRT) and Rasch Measurement Theory (RMT). Methods: The present study was part of a larger multisite, open-label study conducted by the Canadian Biomarker Integration Network in Depression (ClinicalTrials.gov: NCT01655706). Trained raters administered the 32 DID items at each of two visits (MDD: baseline, n=211 and Week 8, n=177; healthy participants: baseline, n=112 and Week 8, n=104). The DID's "grid" structure operationalizes intensity and frequency of each item, with clear symptom definitions and a structured interview guide, with the current iteration assessing symptoms related to anhedonia, cognition, fatigue, general malaise, motivation, anxiety, negative thinking, pain, and appetite. Participants were also administered the Montgomery- Åsberg Depression Rating Scale (MADRS) and Quick Inventory of Depressive Symptomatology-Self-Report (QIDS-SR) that allowed DID items to be evaluated against existing "benchmark" items. CTT was used to assess data quality/reliability (i.e., missing data, skewness, scoring frequency, internal consistency), IRT to assess individual item performance by modelling an item's ability to discriminate levels of depressive severity (as assessed by the MADRS), and RMT to assess how the items perform together as a scale to capture a range of depressive severity (item targeting). These analyses together provided empirical evidence to base decisions on which DID items to remove, modify, or advance. Results: Of the 32 DID items evaluated, eight items were identified by CTT as problematic, displaying low variability in the range of responses, floor effects, and/or skewness; and four items were identified by IRT to show poor discriminative properties that would limit their clinical utility. Five additional items were deemed to be redundant. The remaining 15 DID items all fit the Rasch model, with person and item difficulty estimates indicating satisfactory item targeting, with lower precision in participants with mild levels of depression. These 15 DID items also showed good internal consistency (alpha=0.95 and inter-item correlations ranging from r=0.49 to r=0.84) and all items were sensitive to change following antidepressant treatment (baseline vs. Week 8). RMT revealed problematic item targeting for the MADRS and QIDSSR, including an absence of MADRS items targeting participants with mild/moderate depression and an absence of QIDS-SR items targeting participants with mild or severe depression. Conclusion: The present study applied CTT, IRT, and RMT to assess the measurement properties of the DID items and identify those that should be advanced, modified, or removed. Of the 32 items evaluated, 15 items showed good measurement properties. These items (along with previously evaluated items) will provide the basis for validation of a penultimate DID scale assessing anhedonia, cognitive slowing, concentration, executive function, recent memory, drive, emotional fatigue, guilt, self-esteem, hopelessness, tension, rumination, irritability, reduced appetite, insomnia, sadness, worry, suicidality, and depressed mood. The strategies adopted by the DID process provide a framework for rating scale development and validation.

Sources of unreliability in depression ratings.

BACKGROUND: Good interrater reliability is essential to minimize error variance and improve study power. Reasons why raters differ in scoring the same patient include information variance (different information obtained because of asking different questions), observation variance (the same information is obtained, but raters differ in what they notice and remember), interpretation variance (differences in the significance attached to what is observed), criterion variance (different criteria used to score items), and subject variance (true differences in the subject). We videotaped and transcribed 30 pairs of interviews to examine the most common sources of rater unreliability. METHOD: Thirty patients who experienced depression were independently interviewed by 2 different raters on the same day. Raters provided rationales for their scoring, and independent assessors reviewed the rationales, the interview transcripts, and the videotapes to code the main reason for each discrepancy. One third of the interviews were conducted by raters who had not administered the Hamilton Depression Rating Scale before; one third, by raters who were experienced but not calibrated; and one third, by experienced and calibrated raters. RESULTS: Experienced and calibrated raters had the highest interrater reliability (intraclass correlation [ICC]; r = 0.93) followed by inexperienced raters (r = 0.77) and experienced but uncalibrated raters (r = 0.55). The most common reason for disagreement was interpretation variance (39%), followed by information variance (30%), criterion variance (27%), and observation variance (4%). Experienced and calibrated raters had significantly less criterion variance than the other cohorts (P = 0.001). CONCLUSIONS: Reasons for disagreement varied by level of experience and calibration. Experienced and uncalibrated raters should focus on establishing common conventions, whereas experienced and calibrated raters should focus on fine tuning judgment calls on different thresholds of symptoms. Calibration training seems to improve reliability over experience alone. Experienced raters without cohort calibration had lower reliability than inexperienced raters.

Use of remote centralized raters via live 2-way video in a multicenter clinical trial for schizophrenia.

Factors associated with clinician assessment may play a role in the increasing rate of failed trials. The use of centralized raters, a small group of highly skilled, tightly calibrated, and continuously monitored raters linked to the study sites through videoconferencing can address these issues by reducing the sheer number of raters involved, using rigorous calibration procedures not logistically feasible with a larger dispersed group of raters, and by blinding raters to visit and protocol. This phase 2 study was the first randomized controlled trial to use centralized raters in a study of treatments for schizophrenia. Subjects (N = 313) from 32 sites were randomly assigned to 6 weeks of treatment with 1 of 2 doses of an investigational antipsychotic, olanzapine 15 mg, or placebo. Subjects were evaluated weekly using the Positive and Negative Syndrome Scale. Data from the olanzapine (n = 68) and placebo (n = 68) arms were provided by the sponsor. The mean Positive and Negative Syndrome Scale change was significantly greater with olanzapine (-15.2) than placebo (-4.43), P = 0.002. The significant difference was apparent at week 1. The effect size was 0.48. Internal consistency was high throughout the study. Scores at screening were normally distributed and not skewed toward the cutoff score. Results found that hospitalized patients with schizophrenia were willing and able to participate in clinical trials using remote interviews conducted via videoconference. This methodology shows enormous promise for use in clinical trials, even with acutely psychotic patients.

Face-to-face versus remote administration of the Montgomery-Asberg Depression Rating Scale using videoconference and telephone.

Although the use of telemedicine in psychiatry has a long history in providing clinical care to patients, its use in clinical trials research has not yet been commonly employed. Telemedicine allows for the remote assessment of study patients, which could be done by a centralized, highly calibrated, and impartial cohort of raters independent of the study site. This study examined the comparability of remote administration of the Montgomery-Asberg Depression Rating Scale (MADRS) by videoconference and by telephone to traditional face-to-face administration. Two parallel studies were conducted: one compared face-to-face with videoconference administration (N=35), and the other compared face-to-face with telephone administration (N=35). In each study, depressed patients were interviewed independently twice: once in the traditional face-to-face manner, and the second time by either videoconference or teleconference. A counterbalanced order was used. The mean MADRS score for interviews conducted remotely by videoconference was not significantly different from the mean MADRS scores conducted by face-to-face administration (mean difference=0.51 points), P=.388, intraclass correlation (ICC)=.94, P<0001. Similarly, the mean MADRS score for interviews conducted by telephone was not significantly different from the mean MADRS score conducted by face-to-face administration (mean difference=0.74 points), P=.270, ICC=.93, P<0001. Results of the study support the comparability of remote administration of the MADRS, by both telephone and videoconference, to face-to-face administration. Comparability of the administration mode allows for remote assessment of patients in both research and clinical applications.

The GRID-HAMD: standardization of the Hamilton Depression Rating Scale.

This report describes the GRID-Hamilton Depression Rating Scale (GRID-HAMD), an improved version of the Hamilton Depression Rating Scale that was developed through a broad-based international consensus process. The GRID-HAMD separates the frequency of the symptom from its intensity for most items, refines several problematic anchors, and integrates both a structured interview guide and consensus-derived conventions for all items. Usability was established in a small three-site sample of convenience, evaluating 29 outpatients, with most evaluators finding the scale easy to use. Test-retest (4-week) and interrater reliability were established in 34 adult outpatients with major depressive disorder, as part of an ongoing clinical trial. In a separate study, interrater reliability was found to be superior to the Guy version of the HAMD, and as good as the Structured Interview Guide for the Hamilton Depression Rating Scale (SIGH-D), across 30 interview pairs. Finally, using the SIGH-D as the criterion standard, the GRID-HAMD demonstrated high concurrent validity. Overall, these data suggest that the GRID-HAMD is an improvement over the original Guy version as well as the SIGH-D in its incorporation of innovative features and preservation of high reliability and validity.

A comparison of face-to-face and remote assessment of inter-rater reliability on the Hamilton Depression Rating Scale via videoconferencing.

Poor inter-rater reliability (IRR) is an important methodological factor that may contribute to failed trials. The sheer number of raters at diverse sites in multicenter trials presents a formidable challenge in calibration. Videoconferencing allows for the evaluation of IRR of raters at diverse sites by enabling raters at different sites to each independently interview a common patient. This is a more rigorous test of IRR than passive rating of videotapes. To evaluate the potential impact of videoconferencing on IRR, we compared IRR obtained via videoconference to IRR obtained using face-to-face interviews. Four raters at three different locations were paired using all pair-wise combinations of raters. Using videoconferencing, each paired rater independently conducted an interview with the same patient, who was at a third, central location. Raters were blind to each others' scores. ICC from this cohort (n=22) was not significantly different from the ICC obtained by a cohort using two face-to-face interviews (n=21) (0.90 vs. 0.93, respectively) nor from a cohort using one face-to-face interview and one remote interview (n=21) (0.88). The mean Hamilton Depression Rating Scale (HAMD) scores obtained were not significantly different. There appears to be no loss of signal using remote methods of calibration compared with traditional face-to-face methods.