Assessing Binary Diagnoses of Bio-behavioral Disorders: The Clinical Relevance of Cohen's Kappa.

Cohen's (Educ Psychol Meth. 1960;23:37-40) kappa statistic has been criticized recently for providing low chance-corrected coefficients (<0.40) despite high levels of examiner agreement (e.g., ≥85%). The AC1 statistic (Series. 2002;2:1-9; BMC Med Res Methodol. 2013;13:61), the alpha statistic (J Clin Epidemiol. 1996;49:775-782), the B statistic (BMC Med Res Methodol. 2013;13:97), and the delta statistic (Commun Statist Theory Meth. 2008;37:760-772) were developed to "correct" this "problem." Each of these statistics produces inaccurate results because they either derive from levels of chance agreement that are incorrect (Gwet's AC1 statistic) or fail to correct for chance levels of agreement (the alpha, B, and delta approaches). Implications for diagnoses of psychiatric disorders are critically discussed. The importance of kappa is demonstrated in a number of clinical scenarios.

Sensory features as diagnostic criteria for autism: sensory features in autism.

In this study, we examined the frequency of sensory-related issues as reported by parents in a large sample of school-age adolescents and adults with autism/autism spectrum disorder (ASD) [1] as compared to a group of individuals receiving similar clinical evaluations for developmental/behavioral difficulties but whose final diagnoses were not on the autism spectrum. In no comparison were the features examined predictive of autism or autism spectrum in comparison to the non-ASD sample. Only failure to respond to noises had sensitivity above .75 in the comparison of the broader autism spectrum group, but specificity was poor. While sensory issues are relatively common in autism/ASD, they are also frequent in other disorders. These results question the rationale for including sensory items as a diagnostic criterion for autism.

Functional Outcomes of Children Identified Early in the Developmental Period as at Risk for ASD Utilizing the The Norwegian Mother, Father and Child Cohort Study (MoBa).

Early identification of autism spectrum disorder (ASD) is regarded as crucial for swift access to early intervention and, subsequently, better outcomes later in life. However, current instruments miss large proportions of children who later go on to be diagnosed with ASD, raising a question of what these instruments measure. The present study utilized data from the Norwegian Mother, Father, and Child Cohort Study and the Autism Birth Cohort study to explore the subsequent developmental and diagnostic characteristics of children raising developmental concern on the six-critical discriminative item criterion of the M-CHAT (DFA6) at 18 months of age (N = 834). The DFA6 identified 28.8% of children diagnosed with ASD (N = 163), but 4.4% with language disorder (N = 188) and 81.3% with intellectual disability (N = 32) without ASD. Scoring in the «at-risk¼ range was associated with lower IQ, impaired functional language, and greater severity of autism symptoms whether children had ASD or not.

Evaluating the reliability of multiple assessments of PTSD symptomatology: multiple examiners, one patient.

The objective of this investigation was to assess the inter-examiner reliability of PTSD symptomatology by 12 clinical examiners who evaluated independently a single-case Vietnam-Era veteran, using videotaped clinician interviews with The Clinician Administered PTSD Scale-1 (CAPS-1). A second patient was utilized for cross-validation purposes. Data were analyzed using a specially designed Kappa statistic. In previous reliability assessments of the CAPS-1, a pair of examiners assessed multiple patients, and demonstrated evidence of high reliability and validity. As in previous reliability assessments, reliability was assessed both for frequency and intensity of PTSD symptomatology in both patients. Results indicated that the reliability levels of the CAPS-1 were consistently and almost exclusively in the excellent to perfect levels of inter-examiner agreement, as based upon both global assessments and on a symptom-by-symptom basis. The results of this investigation are interpreted in the broader framework of their applicability to assessing inter-examiner agreement in clinical trials or other large multi-site studies.

A Review Update on Gender Dysphoria and ASD and Response to Corrections.

Here we address the earlier published review by Øien et al. (J Autism Dev Disord, https://doi.org/10.1007/s10803-018-3686-7 , 2018) and the subsequent comments by Zucker and VanderLaan (J Autism Dev Disord, https://doi.org/10.1007/s10803-018-3748-x , 2018). The current commentary is a response to the corrections put forth by the latter authors where aspects related to diagnostic criteria of Gender Dysphoria and corrections to the existence of a control group were noted. Additionally, to highlight that research on Gender Dysphoria and ASD is becoming a very fast and growing field we also present an update of the original review which show a 13% increase of empirical studies published only in the past few months.

Development of the evaluative method for evaluating and determining evidence-based practices in autism.

Although research in autism has grown more sophisticated, the gap between research knowledge and applicability of research in real world settings has grown. There have been a number of different reviews of evidence-based practices of treatments for young children with autism. Reviews which have critically evaluated the empirical evidence have not found any treatments that can be considered evidence-based. Reasons for this shortcoming are explored, and a new method for the evaluation of empirical evidence is provided. Future uses of this evaluative method are provided as well as a discussion of how this tool might aid in narrowing the research to practice gap.

Reliability of the ADI-R: multiple examiners evaluate a single case.

The authors assessed the reliability of the Autism Diagnostic Interview (ADI-R). Seven Clinical Examiners evaluated a three and one half year old female toddler suspected of being on the Autism Spectrum. Examiners showed agreement levels of 94-96% across all items, with weighted kappa (K(w)) between .80 and .88. They were in 100% agreement on 74% of the items; in excellent agreement on 6% of the items (93-96%, with K(w) between .78 and .85); in good agreement on 7% (89-90%, with K(w) between .62 and 0.68); and in fair agreement on 3% (82 - 84%, with K(w) between .40 and .47). For the remaining 10% of ADI-R items, examiners showed poor agreement (50-81% with K(w )between -.67 and .37).

Gender Dysphoria, Sexuality and Autism Spectrum Disorders: A Systematic Map Review.

In this systematic map review, we aimed (1) at identifying studies including the co-occurrence of autism spectrum disorders (ASD) and gender dysphoria (GD) between 1946 and 2018, and (2) to present the papers included in this systematic map review to provide authors in the field of GD, sexuality and ASD with an important database of studies focusing on this very complex topic. The field is of emerging interest, as observed by the large increase of studies over the past decades, especially since 2015. However, future challenges are to be addressed in future studies.

Enhancing the understanding of clinically meaningful results: A clinical research perspective.

Published research often address aspects related to "statistical significance" but fail to address the clinical and practical importance and meaning of results. Our main objectives in this article are to investigate the merit of common measures of Effect Size in statistical research and to highlight the importance of the simple Relative Risk ratio. In this article we present data where we consider two widely utilized effect size measures (Cohen's d and Pearson's r) in relations to relative risk. We conclude that probability analyses of risk surpass the most commonly used statistical approach used in clinical trials today and should thus be the preferred compared to the misuse and misunderstanding of reporting for instance p-values alone.

Clinical Features of Children With Autism Who Passed 18-Month Screening.

OBJECTIVES: We compared sex-stratified developmental and temperamental profiles at 18 months in children screening negative for autism spectrum disorder (ASD) on the Modified Checklist for Autism in Toddlers (M-CHAT) but later receiving diagnoses of ASD (false-negative group) versus those without later ASD diagnoses (true-negative group). METHODS: We included 68 197 screen-negative cases from the Norwegian Mother and Child Cohort Study (49.1% girls). Children were screened by using the 6 critical items of the M-CHAT at 18 months. Groups were compared on domains of the Ages and Stages Questionnaire and the Emotionality Activity Sociability Temperament Survey. RESULTS: Despite passing M-CHAT screening at 18 months, children in the false-negative group exhibited delays in social, communication, and motor skills compared with the true-negative group. Differences were more pronounced in girls. However, with regard to shyness, boys in the false-negative group were rated as more shy than their true-negative counterparts, but girls in the false-negative group were rated as less shy than their counterparts in the true-negative group. CONCLUSIONS: This is the first study to reveal that children who pass M-CHAT screening at 18 months and are later diagnosed with ASD exhibit delays in core social and communication areas as well as fine motor skills at 18 months. Differences appeared to be more pronounced in girls. With these findings, we underscore the need to enhance the understanding of early markers of ASD in boys and girls, as well as factors affecting parental report on early delays and abnormalities, to improve the sensitivity of screening instruments.

Social and communication abilities and disabilities in higher functioning individuals with autism spectrum disorders: the Vineland and the ADOS.

The relationship between adaptive functioning (ability) and autism symptomatology (disability) remains unclear, especially for higher functioning individuals with autism spectrum disorder (ASD). This study investigates ability and disability using the Vineland and Autism Diagnostic Observation Schedule (ADOS), respectively, in two clinical samples of children with ASD. Participants included 187 males with VIQ > 70. Vineland scores were substantially below VIQ, highlighting the magnitude of adaptive impairments despite cognitive potential. A weak relationship was found between ability and disability. Negative relationships were found between age and Vineland scores and no relationships were found between age and ADOS scores. Positive relationships were found between IQ and Vineland Communication. Results stress the need for longitudinal studies on ability and disability in ASD and emphasize the importance of adaptive skills intervention.

"Listen my children and you shall hear": auditory preferences in toddlers with autism spectrum disorders.

PURPOSE: This study tests the hypothesis that toddlers with autism spectrum disorders (ASD) will show differences from contrast groups in preferences for attending to speech. METHOD: This study examined auditory preferences in toddlers with ASD and matched groups of (a) typical age-mates, (b) age-mates with nonautistic developmental disabilities, and (c) younger children matched for language age. The experimental procedure measured time spent oriented to auditory stimuli that were created to exemplify language patterns that had been studied in typically developing infants. RESULTS: Findings suggest that toddlers with ASD show a reduced preference for child-directed speech, compared with typical age-mates, but few differences from children with nonautistic developmental disorders. Correlational analysis revealed that time spent listening to child-directed speech by children with ASD was related to their concurrent receptive language ability as well as to receptive language abilities 1 year later. This relationship did not hold for the other groups. CONCLUSION: The present study supports the hypothesis that children with ASD perform differently from typical peers in auditory preference paradigms and that performance in these tasks is related to concurrent and later language development.

The Quantification of Violence Scale: a simple method of recording significant violence.

BACKGROUND: Although there are many rating scales recording the incidence and intensity of violence there are none that are specifically concerned with the measurement and assessment of severe violence. AIMS: To develop a scale sensitive to variation centred on severe violence, establish its normative values, test its feasibility, and assess its reliability and validity in different populations. METHOD: The Quantification of Violence Scale (QOVS) was developed in two stages. First, a list of 30 commonly experienced violent episodes in clinical psychiatric practice were evaluated and tested by weighting each episode by severity. Second, a numerical scale used to record the severity of the episode according to its degree of planning, intent and consequences. Violent episodes in two clinical populations were compared using the Modified Overt Aggression Scale (MOAS) and the preliminary version of the QOVS over periods up to 18 months, following which the numerical scale was developed. RESULTS: Good (0.60-0.74) to Excellent (> 0.75) test-retest and inter-rater reliability agreement was obtained with both forms of the scale (intra-class correlations of 0.75 and 0.69 respectively), and similar agreement with MOAS scores was reached (0.67) in clinical populations. The scale was quick and easy to use in practice, and a score defining severe violence (9 on the numerical scale and 16 on the matched scale) was determined. CONCLUSIONS: The QOVS, in its two forms, is a useful measure of recording significant violence in clinical and forensic practice.

Risperidone and adaptive behavior in children with autism.

OBJECTIVE: To evaluate the impact of risperidone on adaptive behavior in children with autistic disorder who have serious behavior problems and to examine different methods of scoring the Vineland Adaptive Behavior Scales to measure change. METHOD: Forty-eight children (5 years to 16 years, 5 months) who showed behavioral improvement during acute treatment with risperidone were followed for 6 months and assessed with the Vineland Scales. RESULTS: Raw scores, age-equivalents, and special norm percentile scores all showed significant increases in adaptive behavior in the areas of communication, daily living skills, and socialization (p <.01). During a period of 6 to 8 months, children gained an average of 7.8 age-equivalent months in the area of socialization, a > 6% improvement beyond what would be expected based on baseline growth rates. CONCLUSIONS: Although limited by the absence of a control group, these results suggest that risperidone may improve adaptive skills in children with autistic disorder accompanied by serious behavioral problems. Vineland age-equivalent scores appear to be most useful in assessing change with treatment over time.

The bed requirement inventory: a simple measure to estimate the need for a psychiatric bed.

OBJECTIVE: To develop an assessment of bed need that was as little affected by personal bias as possible. METHOD: The Bed Requirement Inventory (BRI) is an eight-point scale designed to identify the appropriate use of an acute psychiatric bed. This is completed by a member of the ward staff, usually a nurse, and takes 5 minutes to fill in. The reliability, validity and feasibility of using the scale in normal practice were tested in a one-year study, and variations in inappropriate bed use described. RESULTS: The inter-rater reliability of the scale was good (intra-class correlation coefficient = 0.63) and a comparison of the need for a psychiatric bed (comparing the BRI score with the judgement of an independent multidisciplinary group of professionals) also showed good agreement (k = 0.69), suggesting reasonable validity (although when the assessment was made by the named nurse agreement was less good). Results from a year-long survey in two West London hospitals showed that 17% of admissions were inappropriate and 32% had delayed discharge, black Caribbean patients had a significantly higher proportion (25%) of inappropriate admission than others (11%) and those referred from housing charities and hostels had a higher proportion (50%) of inappropriate bed use at some time than other groups (33%). CONCLUSIONS: The Bed Requirement Inventory is a quick and reliable method of determining the appropriate use of a psychiatric bed and could be of use in estimating local bed needs. Delayed discharge remains a serious reason for inappropriate bed use in London.

Is posttraumatic stress in youth a culture-bound phenomenon? A comparison of symptom trends in selected U.S. and Russian communities.

OBJECTIVE: The cross-cultural applicability of the concept of posttraumatic stress was investigated by assessing symptom frequency and levels of comorbid psychopathology in adolescents from the United States and Russia. METHOD: A self-report survey was conducted in representative samples of 2,157 adolescents 14 to 17 years old from urban communities of the United States (N=1,212) and Russia (N=945). RESULTS: In both countries, the levels of all three major clusters of posttraumatic symptoms (reexperiencing, avoidance, and arousal), as well as of internalizing psychopathology, increased along with the level of posttraumatic stress. Expectations about the future had a tendency to decrease with increasing posttraumatic stress. No differences between countries in significant interaction effects for symptom levels were found. CONCLUSIONS: The current findings suggest that posttraumatic symptoms and their associations with other adolescent mental health problems are not culture bound and that the psychological consequences of trauma follow similar dynamics cross-culturally.

Data collection for sexually transmitted disease diagnoses: a comparison of self-report, medical record reviews, and state health department reports.

PURPOSE: To compare three methods of data collection on case ascertainment of past chlamydia or gonorrhea diagnoses. METHODS: Data collection for 361 adolescent females between 1998 and 2000 included: 1) face-to-face interviews; 2) computerized and paper medical record reviews; and 3) chlamydia and gonorrhea reports to the state health department. Statistical methods include latent class and composite reference standard analyses. RESULTS: The estimated prevalence of past diagnoses did not differ significantly by data collection method for chlamydia (20.5%, 23.0%, and 19.7% by self-report, medical record reviews, and state health department reports, respectively) or gonorrhea (4.7%, 6.9%, and 5.5%, respectively) during the 2-year study period. The estimated latent class and composite reference standard prevalences for chlamydia were 23.5% and 26.9%, respectively (p=.04 and p < .01 for differences from self-report alone, respectively). For gonorrhea, the estimated latent class and composite reference standard prevalences were 7.8% and 6.9%, respectively (p < .01 for both differences from self-report alone). Kappa scores for self-report compared with the latent class and composite reference standard prevalences ranged from .67 to .80, and the magnitude of under-reporting ranged from 21% to 47%. CONCLUSIONS: The similar case ascertainment from the three sources separately and high reliability of self-report, coupled with its feasibility and low cost, suggest that self-report is a viable data collection method for STD diagnoses. However, using multiple sources may be preferable when time and resources permit given that under-reporting by self-report is likely to occur (particularly for gonorrhea) and that greater case ascertainment can be achieved.

Brief report: relations between prosodic performance and communication and socialization ratings in high functioning speakers with autism spectrum disorders.

Shriberg et al. [Shriberg, L. et al. (2001). Journal of Speech, Language and Hearing Research, 44, 1097-1115] described prosody-voice features of 30 high functioning speakers with autistic spectrum disorder (ASD) compared to age-matched control speakers. The present study reports additional information on the speakers with ASD, including associations among prosody-voice variables and ratings of communication social abilities. Results suggest that the inappropriate sentential stress and hypernasality previously identified in some of these speakers is related to communication/sociability ratings. These findings and associated trends are interpreted to indicate important links between prosodic performance and social and communicative competence. They suggest the need for careful assessment of inappropriate prosody and voice features in speakers with ASD, and for effective intervention programs aimed at reducing the stigmatization of individuals with these conditions.

Defining the "disruptive" in preschool behavior: what diagnostic observation can teach us.

This paper presents the clinical/developmental framework underlying a new diagnostic observational tool, the Disruptive Behavior Diagnostic Observation Schedule (DB-DOS). The special importance of observation for clinical assessment during the preschool period is delineated. The developmental rationale for a multi-dimensional assessment of disruptive behavior in young children, including problems in modulation of negative affect and low competence is discussed. The ways in which the DB-DOS will elucidate distinctions between normative and atypical behavior during this developmental period via (a) the integration of qualitative and quantitative dimensions of behavior within a clinically-sensitive coding system, (b) the observation of child behavior both within, and outside of, the parent-child context and (c) the use of specially designed tasks to "press" for clinically salient behaviors are addressed. The promise of this new method for yielding a more precise, developmentally based description of the phenotype of early onset disruptive behavior problems and for providing a standardized clinical tool for observational assessment of disruptive behavior in young children is presented. Large-scale validation of the measure is currently underway.