Integrative Consensus: A Systematic Approach to Integrating Comprehensive Assessment Data for Young Children With Behavior Problems.

Comprehensive assessments that include parents and teachers are essential when assessing young children vulnerable to emotional and behavioral problems given the multiple systems and contexts that influence and support optimal development (U. Bronfenbrenner & P. A. Morris, 2006; M. J. Guralnick, 2011). However, more data complicate clinical and educational decision making given the challenge of integrating comprehensive data. We report on initial efforts to develop and apply Integrative Consensus procedures designed to synthesize comprehensive assessment data using developmentally informed guidelines. Mother-teacher dyads (N = 295) reported on disruptive behavior in a sample of 295 low-income 3- to 5-year-olds; one-third referred for disruptive behaviors, one-third nonreferred with behavioral concerns, and one-third nonreferred. Two clinicians trained in Integrative Consensus procedures independently applied the framework, with findings highlighting that children identified as disruptive by Integrative Consensus ratings plus mother or teacher ratings significantly predicted behavior problems and impaired social skills. Children identified as disruptive via Integrative Consensus were 4 times more likely to be rated as impaired by their mother at follow-up than by mother or teacher report. Reliability estimates were high (κ = 0.84), suggesting that the method has promise for identifying young children with behavior problems while systematically integrating comprehensive data.

Preschool children's observed disruptive behavior: variations across sex, interactional context, and disruptive psychopathology.

Sex differences in disruptive behavior and sensitivity to social context are documented, but the intersection between them is rarely examined empirically. This report focuses on sex differences in observed disruptive behavior across interactional contexts and diagnostic status. Preschoolers (n = 327) were classified as nondisruptive (51%), clinically at risk (26%), and disruptive (23%) using parent and teacher reports on developmentally validated measures of disruptive behavior and impairment. Observed disruptive behavior was measured with the Disruptive Behavior Diagnostic Observation Schedule, a developmentally sensitive observational paradigm characterizing variation in preschoolers' disruptive behavior across two interactional contexts (parent and examiner). Repeated measures analyses of variance revealed a three-way interaction of child sex by diagnostic status by interactional context (F = 9.81, p < .001). Disruptive boys were the only subgroup whose behavior was not sensitive to interactional context: They displayed comparable levels of disruptive behavior with parents and examiners. In contrast, disruptive girls demonstrated the strongest context effect of any group. Specifically, with the examiner, disruptive girls' behavior was comparable to nondisruptive boys (though still more elevated than nondisruptive girls). However, in interactions with their mothers, disruptive girls displayed the highest rates of disruptive behavior of any subgroup in any context, although the difference between disruptive boys and disruptive girls in this context was not statistically significant. Findings suggest the importance of sex-specific conceptualizations of disruptive behavior in young children that take patterns across social contexts into account.

Predictive validity of DSM-IV oppositional defiant and conduct disorders in clinically referred preschoolers.

BACKGROUND: Diagnostic validity of oppositional defiant and conduct disorders (ODD and CD) for preschoolers has been questioned based on concerns regarding the ability to differentiate normative, transient disruptive behavior from clinical symptoms. Data on concurrent validity have accumulated, but predictive validity is limited. Predictive validity is critical to refuting the hypothesis that diagnosing ODD and CD in young children leads to pathologizing normal behavior. ODD and CD have emerged as gateway disorders to many forms of adult psychopathology. Establishing how early we can identify symptoms and disorders that herald poor prognosis is one of the most important goals for research on etiology and prevention. METHODS: Subjects were 3-5-year-old consecutive referrals to a child psychiatry clinic (n=123) and demographically matched children from a pediatric clinic (n=100). A diagnostic interview was used to assess DSM-IV ODD and CD in a prospective follow-up design from preschool to school age. Stability of ODD and CD diagnoses and level of impairment were tested as a function of preschool diagnosis. RESULTS: Over 80% of preschoolers diagnosed with ODD and approximately 60% of preschoolers diagnosed with CD met criteria for the same disorder during follow-up. Impairment over time varied significantly as a function of stability of diagnosis across three years. CONCLUSIONS: These results provide the first evidence of the predictive validity of DSM-IV ODD and CD in clinically referred preschool children. The findings challenge the assumption that symptoms of disruptive behavior disorders that occur during the preschool period tend to be transient.

Measuring implementation fidelity in a community-based parenting intervention.

BACKGROUND: Establishing the feasibility and validity of implementation fidelity monitoring strategies is an important methodological step in implementing evidence-based interventions on a large scale. OBJECTIVES: The objective of the study was to examine the reliability and validity of the Fidelity Checklist, a measure designed to assess group leader adherence and competence delivering a parent training intervention (the Chicago Parent Program) in child care centers serving low-income families. METHOD: The sample included 9 parent groups (12 group sessions each), 12 group leaders, and 103 parents. Independent raters reviewed 106 audiotaped parent group sessions and coded group leaders' fidelity on the Adherence and Competence Scales of the Fidelity Checklist. Group leaders completed self-report adherence checklists and a measure of parent engagement in the intervention. Parents completed measures of consumer satisfaction and child behavior. RESULTS: High interrater agreement (Adherence Scale = 94%, Competence Scale = 85%) and adequate intraclass correlation coefficients (Adherence Scale = .69, Competence Scale = .91) were achieved for the Fidelity Checklist. Group leader adherence changed over time, but competence remained stable. Agreement between group leader self-report and independent ratings on the Adherence Scale was 85%; disagreements were more frequently due to positive bias in group leader self-report. Positive correlations were found between group leader adherence and parent attendance and engagement in the intervention and between group leader competence and parent satisfaction. Although child behavior problems improved, improvements were not related to fidelity. DISCUSSION: The results suggest that the Fidelity Checklist is a feasible, reliable, and valid measure of group leader implementation fidelity in a group-based parenting intervention. Future research will be focused on testing the Fidelity Checklist with diverse and larger samples and generalizing to other group-based interventions using a similar intervention model.

Implementation fidelity in community-based interventions.

Implementation fidelity is the degree to which an intervention is delivered as intended and is critical to successful translation of evidence-based interventions into practice. Diminished fidelity may be why interventions that work well in highly controlled trials may fail to yield the same outcomes when applied in real life contexts. The purpose of this paper is to define implementation fidelity and describe its importance for the larger science of implementation, discuss data collection methods and current efforts in measuring implementation fidelity in community-based prevention interventions, and present future research directions for measuring implementation fidelity that will advance implementation science.

Viewing preschool disruptive behavior disorders and attention-deficit/hyperactivity disorder through a developmental lens: what we know and what we need to know.

Empirical investigation into disruptive behavior disorders (DBDs) and attention deficit/hyperactivity disorder (ADHD) in early childhood has expanded considerably during the past decade. Although there have been considerable gains in the understanding of the presentation and course of these psychiatric disorders in early childhood, the lack of a developmental framework to guide nosologic issues likely impedes progress in this area. The authors propose that enhanced developmental sensitivity in defining symptoms of DBDs and ADHD may shed light on outstanding issues in the field. In particular, developmental specification may enhance specificity, sensitivity, and stability of DBDs and ADHD symptoms as well as inform our understanding of which type of treatment works best for whom. This article provides an overview of these critical issues.

Understanding disruptive behavior problems in preschool children.

Disruptive behavior problems in young children are the number one reason for referral to mental health agencies. However, owing to difficulties differentiating clinically significant disruptive behaviors from typical development, a significant proportion of young children with disruptive behavior problems go unidentified and untreated. Research supports the existence of disruptive behavior disorders in young children, and early identification and treatment are critical to interrupt the trajectory of early problems to more significant and impairing difficulties. The purposes of this article were to identify and discuss disruptive behavior problems in preschool children and to introduce readers to current definitions of disruptive behavior problems and emotion regulation. A review of risk factors and underlying emotion and behavior regulation difficulties implicated in the development of disruptive behavior problems is provided. Furthermore, clinical implications for nurses in the identification of disruptive behavior problems in preschool children for pediatric nurses are discussed.

Observational Assessment of Preschool Disruptive Behavior, Part II: validity of the Disruptive Behavior Diagnostic Observation Schedule (DB-DOS).

OBJECTIVE: To examine the validity of the Disruptive Behavior Diagnostic Observation Schedule (DB-DOS), a new observational method for assessing preschool disruptive behavior. METHOD: A total of 327 behaviorally heterogeneous preschoolers from low-income environments comprised the validation sample. Parent and teacher reports were used to identify children with clinically significant disruptive behavior. The DB-DOS assessed observed disruptive behavior in two domains, problems in Behavioral Regulation and Anger Modulation, across three interactional contexts: Examiner Engaged, Examiner Busy, and Parent. Convergent and divergent validity of the DB-DOS were tested in relation to parent and teacher reports and independently observed behavior. Clinical validity was tested in terms of criterion and incremental validity of the DB-DOS for discriminating disruptive behavior status and impairment, concurrently and longitudinally. RESULTS: DB-DOS scores were significantly associated with reported and independently observed behavior in a theoretically meaningful fashion. Scores from both DB-DOS domains and each of the three DB-DOS contexts contributed uniquely to discrimination of disruptive behavior status, concurrently and predictively. Observed behavior on the DB-DOS also contributed incrementally to prediction of impairment over time, beyond variance explained by meeting DSM-IV disruptive behavior disorder symptom criteria based on parent/teacher report. CONCLUSIONS: The multidomain, multicontext approach of the DB-DOS is a valid method for direct assessment of preschool disruptive behavior. This approach shows promise for enhancing accurate identification of clinically significant disruptive behavior in young children and for characterizing subtypes in a manner that can directly inform etiological and intervention research.

Observational Assessment of Preschool Disruptive Behavior, Part I: reliability of the Disruptive Behavior Diagnostic Observation Schedule (DB-DOS).

OBJECTIVE: To examine the reliability of the Disruptive Behavior Diagnostic Observation Schedule (DB-DOS), a new observational method for assessing preschool disruptive behavior. METHOD: The DB-DOS is a structured clinic-based assessment designed to elicit clinically salient behaviors relevant to the diagnosis of disruptive behavior in preschoolers. Child behavior is assessed in three interactional contexts that vary by partner (parent versus examiner) and level of support provided. Twenty-one disruptive behaviors are coded within two domains: problems in Behavioral Regulation and problems in Anger Modulation. A total of 364 referred and nonreferred preschoolers participated: interrater reliability and internal consistency were assessed on a primary sample (n = 335) and test-retest reliability was assessed in a separate sample (n = 29). RESULTS: The DB-DOS demonstrated good interrater and test-retest reliability. Confirmatory factor analysis demonstrated an excellent fit of the DB-DOS multidomain model of disruptive behavior. CONCLUSIONS: The DB-DOS is a reliable observational tool for clinic-based assessment of preschool disruptive behavior. This standardized assessment method holds promise for advancing developmentally sensitive characterization of preschool psychopathology.

Validation of a Clinically Sensitive, Observational Coding System for Parenting Behaviors: The Parenting Clinical Observation Schedule.

Objective: The psychometric properties of a newly developed, clinically-sensitive, observational measure of parenting behavior, the Parenting Clinical Observational System (P-COS), are presented. The P-COS expands on existing observational coding systems by integrating qualitative and quantitative aspects of parenting behavior to directly inform clinical decision-making with parents of young children with disruptive behaviors. Design: Reliability and validity data were examined in a sample of mother-preschooler dyads (N = 335), and test-re-test reliability was assessed on an independent sample (N = 29). Results: Evidence was found for the reliability and validity of the P-COS, including evidence of inter-rater reliability, internal consistency, and distinctness of the three theorized P-COS domains: Responsive Involvement, Constructive Discipline, and Problematic Discipline. Convergent and divergent validity data are presented as well. The P-COS demonstrated incremental validity in predicting child disruptive behavior over and above self-reported parenting and was associated with change in child clinical status over time. Conclusions: Clinically informative observation methods that assess multiple dimensions of parenting within the context of a child disruptive behavior assessment provide useful information for directing treatment efforts.

A developmental framework for distinguishing disruptive behavior from normative misbehavior in preschool children.

BACKGROUND: Attaining a developmentally sensitive nosology for preschool disruptive behavior requires characterization of the features that distinguish it from the normative misbehavior of this developmental period. We hypothesize that quality of behavior and its pervasiveness across contexts are critical dimensions for clinical discrimination in young children and propose that structured diagnostic observation provides a systematic method for their identification. We use the Disruptive Behavior Diagnostic Observation Schedule (DB-DOS) to examine whether: (a) observed quality and pervasiveness of behavior distinguishes preschoolers with clinically concerning disruptive behavior from typically developing preschoolers, and (b) observed pattern of clinically salient behavior predicts impairment above and beyond maternal report of behavioral frequency. METHODS: Participants are a behaviorally heterogeneous sample of preschoolers (N = 327). Diagnostic methods developed for clinical assessment of preschoolers were used to classify children as (a) Non-Disruptive, (b) Sub-Clinical, or (c) Disruptive. Child behavior was coded based on interactions with parent and examiner during the DB-DOS. RESULTS: Quality and pervasiveness of observed behaviors during the DB-DOS significantly distinguished the three behavioral groups. Discriminative utility varied depending on the comparison. With few exceptions, clinically concerning patterns on the DB-DOS added significant incremental utility in predicting impairment. CONCLUSIONS: Observed patterns of clinically salient behavior show promise for advancing developmentally-informed characterization of disruptive behavior within the preschool period.

Further evidence of the reliability and validity of DSM-IV ODD and CD in preschool children.

OBJECTIVE: To test the reliability and validity of DSM-IV oppositional defiant and conduct disorders (ODD and CD) and symptoms using the Kiddie Disruptive Behavior Disorders Schedule and generate data on the manifestation of symptoms of ODD and CD in 3- to 5-year-old children. METHOD: One hundred twenty-three consecutive referrals to a child and adolescent psychiatry clinic and 100 children recruited from a pediatric clinic and their biological mothers participated. Mothers were interviewed about current ODD and CD symptoms, and mother and teachers completed measures of impairment. Twenty-five percent of the interviews were randomly selected and coded by a second rater to test interrater reliability. A separate sample of 31 children was recruited for a 1-week test-retest study. RESULTS: Overall, reliability of ODD and CD symptoms as measured by interrater and test-retest reliability was moderate to high. Reliability was not affected by age or sex of the child. Validity was demonstrated via associations with impairment and differentiation between referred and nonreferred children. CONCLUSIONS: Most DSM-IV ODD and CD symptoms can be reliably and validly assessed in preschool children. Some developmental modifications are recommended. The results add to the literature on best methods of assessment of preschool psychopathology.

Mother's childrearing history and current parenting: patterns of association and the moderating role of current life stress.

This study examined the association between positive and negative aspects of childrearing history and current parenting and the moderating effect of current stress. Seventy mother-child dyads participated in this study. Mothers provided retrospective reports of childrearing histories and current reports of life stress. Parenting was assessed via maternal self-report and observation. There was modest support for a direct association between positive childrearing experiences and more positive current parenting. Stress moderated the associations between both positive and negative childrearing experiences and current parenting: Stress exacerbated the negative effects of high-conflict histories whereas positive histories protected against the effects of current stress. This study highlights the importance of studying the influence of early experience on parenting within the context of current life stress.

Defining the "disruptive" in preschool behavior: what diagnostic observation can teach us.

This paper presents the clinical/developmental framework underlying a new diagnostic observational tool, the Disruptive Behavior Diagnostic Observation Schedule (DB-DOS). The special importance of observation for clinical assessment during the preschool period is delineated. The developmental rationale for a multi-dimensional assessment of disruptive behavior in young children, including problems in modulation of negative affect and low competence is discussed. The ways in which the DB-DOS will elucidate distinctions between normative and atypical behavior during this developmental period via (a) the integration of qualitative and quantitative dimensions of behavior within a clinically-sensitive coding system, (b) the observation of child behavior both within, and outside of, the parent-child context and (c) the use of specially designed tasks to "press" for clinically salient behaviors are addressed. The promise of this new method for yielding a more precise, developmentally based description of the phenotype of early onset disruptive behavior problems and for providing a standardized clinical tool for observational assessment of disruptive behavior in young children is presented. Large-scale validation of the measure is currently underway.