Caregiver skills training for caregivers of individuals with neurodevelopmental disorders: A systematic review and meta-analysis.

AIM: To systematically review the effectiveness of caregiver and parent skills training programs, including caregiver-mediated interventions, for caregivers of individuals with neurodevelopmental disorders. METHOD: We conducted a systematic review with a random-effects meta-analysis. We searched 11 electronic databases through July 2021 and used a snowball methodology to locate relevant articles of randomized controlled trials. Effect size estimates were pooled using Hedges' g from data extracted from study reports and through author requests using random-effects meta-analyses for three child outcome categories (child development, adaptive behavior, and problem behavior) and three caregiver outcome categories (parenting skills and knowledge, psychological well-being, and interpersonal family relations). RESULTS: We located 44 910 records, from which 75 randomized controlled trials involving 4746 individuals with neurodevelopmental disorders and their caregivers were included. Random-effects meta-analyses showed improvements in child development (g = 0.30; 99% confidence interval [CI] = 0.07-0.53) and reduction in reported problem behaviors (g = 0.41; 99% CI = 0.24-0.59), but not a statistically significant improvement in adaptive behavior (g = 0.28; 99% CI = -0.42 to 0.98). Caregivers showed improvements in parenting skills and knowledge (g = 0.72; 99% CI = 0.53-0.90), psychological well-being (g = 0.52; 99% CI = 0.34-0.71), and interpersonal family relations (g = 0.76; 99% CI = 0.32-1.20). INTERPRETATION: Caregiver skills training programs benefit both caregivers and children with neurodevelopmental disorders. Skills training programs improve child development and behavior, improve parenting skills, reduce caregiver mental health issues, and improve family functioning. Programs using culturally appropriate training material to improve the development, functioning, and participation of children within families and communities should be considered when caring for children with neurodevelopmental disorders.

Beginning reading interventions for children and adolescents with intellectual disability.

BACKGROUND: Historically, students with intellectual disability were not expected to learn to read, and thus were excluded from reading instruction. Over the past decades, societal expectations for this group of learners have changed in that children and adolescents with intellectual disability are now expected to be provided with, and benefit from, literacy instruction. This shift in societal expectations has also led to an increase in research examining effective interventions for increasing beginning reading skills for students with intellectual disability. OBJECTIVES: To assess the effectiveness of interventions for teaching beginning reading skills to children and adolescents with intellectual disability. SEARCH METHODS: We searched the following electronic databases up to October 2019: CENTRAL; MEDLINE, including Epub Ahead of Print and In-Process and Other Non-Indexed Citations, Embase, 13 other databases, and two trials registers. We contacted authors of included studies, examined reference lists, and used Google Scholar to search for additional studies. SELECTION CRITERIA: We included randomized controlled trials (including trials that use quasi-random methods of allocation such as date of birth), involving children and adolescents with intellectual disability (defined as an intelligence quotient (IQ) two standard deviations or more below the population mean) between the ages of 4 and 21 years, that evaluated the efficacy of a beginning reading intervention compared to a control intervention, including no treatment control, wait-list control, treatment as usual, attention control, or alternate non-reading instruction control. DATA COLLECTION AND ANALYSIS: Two review authors independently screened titles and abstracts yielded by the search against the inclusion criteria, and extracted data from each trial using a piloted data extraction form to collect information about the population, intervention, randomization methods, blinding, sample size, outcome measures, follow-up duration, attrition and handling of missing data, and methods of analysis. When data were missing, one review author contacted the study authors to request additional information. Two review authors assessed the risk of bias of each included study and rated the quality of the evidence using the GRADE approach (a systematic method for rating the certainty of evidence in meta-analyses). We conducted random-effect meta-analyses, with inverse-variance weighting to combine effect sizes for each of our primary and secondary outcomes. We presented effect sizes as standardized mean differences (SMD) with 95% confidence intervals (CI). MAIN RESULTS: We identified seven studies involving 352 children and adolescents with intellectual disabilities that met the inclusion criteria. All studies provided the intervention in school settings. Four studies were conducted in the USA, one in Canada, and two in the UK. Three studies were funded by grants from the US Department of Education, Institute of Education Sciences; one study by the Canadian Language and Literacy Research Network and the Nova Scotia Health Research Foundation; and three studies did not indicate a funding source. We identified some concerns with risk of bias, mainly due to the difficulty of blinding of participants and personnel, and the lack of blinding of outcome assessors. Meta-analyses of the data demonstrated small-to-moderate effects of beginning reading interventions delivered to children and adolescents with intellectual disability across four dependent variables. We found medium effect sizes in favor of the beginning reading interventions for the primary outcomes of phonologic awareness (SMD 0.55, 95% CI 0.23 to 0.86; 4 studies, 178 participants; moderate-quality evidence), word reading (SMD 0.54, 95% CI 0.05 to 1.03; 5 studies, 220 participants; moderate-quality evidence), and decoding (SMD 0.40, 95% CI 0.12 to 0.67; 5 studies, 230 participants; low-quality evidence). The studies reported no adverse events. We also found a moderate effect for the secondary outcomes of oral reading fluency (SMD 0.65, 95% CI -0.12 to 1.42; 2 studies, 84 participants; low-quality evidence) and language skills (SMD 0.28, 95% CI 0.03 to 0.54; 3 studies, 222 participants; moderate-quality evidence). AUTHORS' CONCLUSIONS: Results from this review provide evidence that beginning reading interventions that include elements of phonologic awareness, letter sound instruction, and decoding, delivered to children and adolescents with intellectual disability, are associated with small-to-moderate improvements in phonologic awareness, word reading, decoding, expressive and receptive language, and oral reading fluency. These findings are aligned with previously conducted studies that examined the effects of reading interventions for people without intellectual disability.

Development and applications of the single-case design risk of bias tool for evaluating single-case design research study reports.

Systematic reviews and meta-analyses can be a useful method for synthesizing evidence across multiple studies to draw conclusions about a research base. An important aspect of rigorous systematic reviews is an assessment of the study methods and potential biases impacting results or interpretations and conclusions of the primary studies. Single-case design (SCD) research has been a primary mechanism for identifying evidence-based practices across disciplines, but primarily in behavioral fields and special education. While the Cochrane Risk of Bias tool has been adapted for use in reviews of non-randomized studies, there is currently no guidance for evaluating SCD research. Hence, we developed a single case design risk of bias tool (SCD RoB) based on current conceptualizations of biases that might affect the validity of claims from single-case design research. We used the Cochrane risk of bias criteria and contemporary single-case design quality indicators and design standards to guide development. We describe the SCD RoB tool and two early applications of its use to demonstrate its application and provide initial validation. We also provide an overview of future areas of research using the SCD RoB tool in an effort to advance the science of single-case design research methods.

Reliability, Validity, and Usability of Data Extraction Programs for Single-Case Research Designs.

Single-case experimental designs (SCEDs) have been increasingly used in recent years to inform the development and validation of effective interventions in the behavioral sciences. An important aspect of this work has been the extension of meta-analytic and other statistical innovations to SCED data. Standard practice within SCED methods is to display data graphically, which requires subsequent users to extract the data, either manually or using data extraction programs. Previous research has examined issues of reliability and validity of data extraction programs in the past, but typically at an aggregate level. Little is known, however, about the coding of individual data points. We focused on four different software programs that can be used for this purpose (i.e., Ungraph, DataThief, WebPlotDigitizer, and XYit), and examined the reliability of numeric coding, the validity compared with real data, and overall program usability. This study indicates that the reliability and validity of the retrieved data are independent of the specific software program, but are dependent on the individual single-case study graphs. Differences were found in program usability in terms of user friendliness, data retrieval time, and license costs. Ungraph and WebPlotDigitizer received the highest usability scores. DataThief was perceived as unacceptable and the time needed to retrieve the data was double that of the other three programs. WebPlotDigitizer was the only program free to use. As a consequence, WebPlotDigitizer turned out to be the best option in terms of usability, time to retrieve the data, and costs, although the usability scores of Ungraph were also strong.

Curriculum-based measurement of oral reading (R-CBM): a diagnostic test accuracy meta-analysis of evidence supporting use in universal screening.

A great deal of research over the past decade has examined the appropriateness of curriculum-based measurement of oral reading (R-CBM) in universal screening. Multiple researchers have meta-analyzed available correlational evidence, yielding support for the interpretation of R-CBM as an indicator of general reading proficiency. In contrast, researchers have yet to synthesize diagnostic accuracy evidence, which pertains to the defensibility of the use of R-CBM for screening purposes. The overall purpose of this research was to therefore conduct the first meta-analysis of R-CBM diagnostic accuracy research. A systematic search of the literature resulted in the identification of 34 studies, including 20 peer-reviewed articles, 7 dissertations, and 7 technical reports. Bivariate hierarchical linear models yielded generalized estimates of diagnostic accuracy statistics, which predominantly exceeded standards for acceptable universal screener performance. For instance, when predicting criterion outcomes within a school year (≤9 months), R-CBM sensitivity ranged between .80 and .83 and specificity ranged between .71 and .73. Multiple moderators of R-CBM diagnostic accuracy were identified, including the (a) R-CBM cut score used to define risk, (b) lag in time between R-CBM and criterion test administration, and (c) percentile rank corresponding to the criterion test cut score through which students were identified as either truly at risk or not at risk. Follow-up analyses revealed substantial variability of extracted cut scores within grade and time of year (i.e., fall, winter, and spring). This result called into question the inflexible application of a single cut score across contexts and suggested the potential necessity of local cut scores. Implications for practices, directions for future research, and limitations are discussed.

Evaluating single-case research data for systematic review: a commentary for the special issue.

The purpose of this commentary is to provide observation on the statistical procedures described throughout this special section from the perspective of researchers with experience in conducting systematic reviews and meta-analyses of single-case research to address issues of evidence-based practice. It is our position that both visual and statistical analyses are complimentary methods for evaluating single-case research data for these purposes. Given the recent developments regarding the use of single-case research to inform evidence-based practice and policy, the developments described in the present issue will be contextualized within the need for a widely accepted process for data evaluation to assist with extending the impact of single-case research. The commentary will, therefore, begin with providing an overview of the conceptual underpinnings of a systematic review of single-case research and will be followed by a discussion of several features that are essential to the development of a conceptually sound and widely used statistical procedure for single-case research. The commentary will conclude with recommendations and guidelines for the use of both visual and statistical analyses within primary research reports and recommendations for future research.

A systematic evidence review of school-based group contingency interventions for students with challenging behavior.

The purpose of this review was to synthesize the research underlying group contingency interventions to determine whether there is sufficient evidence to support their use for managing the classroom behavior of students with behavioral difficulties. An application of the What Works Clearinghouse (WWC) procedures for evaluating single-subject research revealed that the research investigating group contingencies demonstrated sufficient rigor, evidence, and replication to label the intervention as evidence-based. These findings were further supported across five quantitative indices of treatment effect. The results associated with the application of the WWC procedures and quantitative evaluations were supplemented with additional systematic coding of methodological features and study characteristics to evaluate the populations and conditions under which the effects of the group contingency best generalize. Findings associated with this coding revealed that the lack of detailed reporting across studies limited our ability to determine for whom and under what conditions group contingencies are best suited.

A systematic evaluation of token economies as a classroom management tool for students with challenging behavior.

A two-part systematic review was undertaken to assess the effectiveness of token economies in increasing rates of appropriate classroom behavior for students demonstrating behavioral difficulties. The first part of the review utilized the recently published What Works Clearinghouse (WWC) standards for evaluating single-subject research to determine the extent to which eligible studies demonstrated sufficient evidence to classify the token economy as an evidence-based practice. The second part of the review employed meta-analytic techniques across four different types of effect sizes to evaluate the quantitative strength of the findings. Methodological strengths and weaknesses across the studies were systematically investigated. Results indicated that the extant research on token economies does not provide sufficient evidence to be deemed best-practice based on the WWC criteria.

A generalized least squares regression approach for computing effect sizes in single-case research: application examples.

A new method for deriving effect sizes from single-case designs is proposed. The strategy is applicable to small-sample time-series data with autoregressive errors. The method uses Generalized Least Squares (GLS) to model the autocorrelation of the data and estimate regression parameters to produce an effect size that represents the magnitude of treatment effect from baseline to treatment phases in standard deviation units. In this paper, the method is applied to two published examples using common single case designs (i.e., withdrawal and multiple-baseline). The results from these studies are described, and the method is compared to ten desirable criteria for single-case effect sizes. Based on the results of this application, we conclude with observations about the use of GLS as a support to visual analysis, provide recommendations for future research, and describe implications for practice.