The Interrater Reliability of the Autism Diagnostic Interview-Revised (ADI-R) in Clinical Settings.

BACKGROUND: The Autism Diagnostic Interview-Revised (ADI-R) is considered a first choice assessment tool in autism spectrum disorder. Nevertheless, despite its wide use in psychiatric practice and recommendations by various clinical guidelines, its interrater reliability has predominantly been confirmed in research settings by specially trained, research reliability interviewers. The reliability of ADI-R assessments among clinicians has not yet been established. Therefore, this study examined the spontaneous interrater reliability of the ADI-R in a naturalistic clinical multicenter setting. SAMPLING AND METHODS: Ten video-recorded ADI-R administrations were rated by 5 different raters each from a pool of 11 raters affiliated to 8 different clinical sites. RESULTS: The interrater reliability for the 12 diagnostic criteria operationalizing autism spectrum disorders according to DSM-IV/ICD-10 in the ADI-R algorithms ranged between G(q,k) (analogous to intraclass correlations) = 0.96 and 0.99 for reciprocal social interaction, 0.96 and 1.00 for communication, and 0.91 and 0.97 for repetitive and restricted behavior. Reliability of diagnostic classification was ĸCohen 0.83. CONCLUSIONS: The findings endorse the psychometric properties of ADI-R in terms of interrater reliability previously reported from research settings and support their generalization to common clinical settings. Limitations of this study include an unbalanced sample composition.

The objectivity of the Autism Diagnostic Observation Schedule (ADOS) in naturalistic clinical settings.

The Autism Diagnostic Observation Schedule (ADOS) is a first-choice diagnostic tool in autism spectrum disorder (ASD). Excellent interpersonal objectivity (interrater reliability) has been demonstrated for the ADOS under optimal conditions, i.e., within groups of highly trained "research reliable" examiners in research setting. We investigated the spontaneous interrater reliability among clinically trained ADOS users across multiple sites in clinical routine. Forty videotaped administrations of the ADOS modules 1-4 were rated by five different raters each from a pool of in total 15 raters affiliated to 13 different clinical sites. G(q,k) coefficients (analogous to intraclass correlations), kappas (ĸ) and percent agreement (PA) were calculated. The median interrater reliability for items across the four modules was G(q,k) = .74-.83, with the single ADOS items ranging from .23 to .94. G(q,k) for total scores was .85-.92. For diagnostic classification (ASD/non-spectrum), PA was 64-82 % and Fleiss' ĸ .19-.55. Objectivity was lower for pervasive developmental disorder not otherwise specified and non-spectrum diagnoses as compared to autism. Interrater reliabilities of the ADOS items and domain totals among clinical users across multiple sites were in the same range as previously reported for research reliable users, while the one for diagnostic classification was lower. Differences in sample characteristics, rater skills and statistics compared with previous studies are discussed. Findings endorse the objectivity of the ADOS in naturalistic clinical settings, but also pinpoint its limitations and the need and value of adequate and continuous rater training.