Meta-Analytic Methods to Detect Publication Bias in Behavior Science Research.

Publication bias is an issue of great concern across a range of scientific fields. Although less documented in the behavior science fields, there is a need to explore viable methods for evaluating publication bias, in particular for studies based on single-case experimental design logic. Although publication bias is often detected by examining differences between meta-analytic effect sizes for published and grey studies, difficulties identifying the extent of grey studies within a particular research corpus present several challenges. We describe in this article several meta-analytic techniques for examining publication bias when published and grey literature are available as well as alternative meta-analytic techniques when grey literature is inaccessible. Although the majority of these methods have primarily been applied to meta-analyses of group design studies, our aim is to provide preliminary guidance for behavior scientists who might use or adapt these techniques for evaluating publication bias. We provide sample data sets and R scripts to follow along with the statistical analysis in hope that an increased understanding of publication bias and respective techniques will help researchers understand the extent to which it is a problem in behavior science research.

Meta-analyses and effect sizes in applied behavior analysis: A review and discussion.

For more than four decades, researchers have used meta-analyses to synthesize data from multiple experimental studies often to draw conclusions that are not supported by individual studies. More recently, single-case experimental design (SCED) researchers have adopted meta-analysis techniques to answer research questions with data gleaned from SCED experiments. Meta-analyses enable researchers to answer questions regarding intervention efficacy, generality, and condition boundaries. Here we discuss meta-analysis techniques, the rationale for their adaptation with SCED studies, and current indices used to quantify the effect of SCED data in applied behavior analysis.

Systematic Review of Verbal Operants in Speech Generating Device Research from Skinner's Analysis of Verbal Behavior.

Skinner's (1957) book Verbal Behavior is a critical tool in designing effective communication programs for individuals with limited speech. The purpose of this systematic review was to analyze the speech generating device (SGD) research literature from Skinner's taxonomy of primary verbal operants. An extraction procedure yielded 56 studies published between 1995 and 2018, with a total of 221 participants, most of whom had autism spectrum disorder (ASD) or an intellectual and developmental disability (IDD). The large majority of SGD studies (42) targeted multiply controlled mands, whereas only a handful of studies targeted verbal operants that were not mands. Few studies employed procedures for fading contrived sources of stimulus control to promote spontaneous responding, and few studies targeted more sophisticated, topography-based responses (e.g., typing, speech). Results of the review highlight the need for better dissemination of Skinner's Verbal Behavior, the need for research to evaluate effects of SGD in teaching a greater variety of spontaneous verbal operants, and the need to focus on application of SGD with populations beyond individuals with ASD and IDD.

Do we need to go beyond a purely behavioral approach? A response to Woodcock and Blackwell.

: In their recent review, 'Psychological treatment strategies for challenging behaviours in neurodevelopmental disorders: what lies beyond a purely behavioural approach?' Woodcock and Blackwell discuss supposed deficiencies in applied behavior analysis (ABA) and positive behavior support (PBS) to address challenging behavior. This commentary highlights problems with Woodcock and Blackwell's critique based on substantial evidence that behavioral approaches including ABA and PBS are often the best available solution for individuals with neurodevelopmental disorders who display challenging behavior.

Reducing an Adolescent's Destructive Behavior during Bathroom Visits in a Residential Setting: Brief Report.

As individuals with autism spectrum disorder (ASD) transition to adulthood, problem behavior may present restrictions to an independent lifestyle. In addition, problem behavior may interfere with performance of daily routines necessary for independent living. We developed a successful treatment for automatically reinforced toilet lid destruction of an adolescent with ASD during routine bathroom visits. The treatment was informed by a competing stimulus assessment (CSA) that identifies a preferred stimulus, the presentation of which is intended to compete with automatic reinforcement of problem behavior. Results indicated that access to the competing stimulus reduced the participant's toilet lid destruction while in the bathroom, and these treatment gains maintained during long term follow-up visits.

Evaluation of publication bias in response interruption and redirection: A meta-analysis.

Publication bias is the disproportionate representation of studies with large effects and statistically significant findings in the published research literature. If publication bias occurs in single-case research design studies on applied behavior-analytic (ABA) interventions, it can result in inflated estimates of ABA intervention effects. We conducted an empirical evaluation of publication bias on an evidence-based ABA intervention for children diagnosed with autism spectrum disorder, response interruption and redirection (RIRD). We determined effect size estimates for published and unpublished studies using 3 metrics, percentage of nonoverlapping data (PND), Hedges' g, and log response ratios (LRR). Omnibus effect size estimates across all 3 metrics were positive, supporting that RIRD is an effective treatment for reducing problem behavior maintained by nonsocial consequences. We observed larger PND for published compared to unpublished studies, small and nonsignificant differences in LRR for published compared to unpublished studies, and significant differences in Hedges' g for published compared to unpublished studies, with published studies showing slightly larger effect. We found little, if any, difference in methodological quality between published and unpublished studies. While RIRD appears to be an effective intervention for challenging behavior maintained by nonsocial consequences, our results reflect some degree of publication bias present in the RIRD research literature.

Assessment and treatment of high-risk challenging behavior of adolescents with autism in an aquatic setting.

Aquatic-based activities produce positive skill and health benefits for individuals with Autism Spectrum Disorder (ASD); however, aquatic contexts, such as the pool, introduce the risk of injury and drowning. This risk is heightened when individuals with ASD engage in challenging behavior in the pool context. The purpose of the study was to evaluate the effects of differential reinforcement without extinction for 2 participants diagnosed with ASD who engaged in challenging behavior when asked to transition from the pool. The treatment successfully decreased participants' transition latencies and challenging behavior during transitions from the pool for up to 2 months following treatment. Lifeguard staff rated the procedures as highly acceptable and helpful, and noted high degrees of satisfaction with improvements for each participant's behavior.

Replication Research, Publication Bias, and Applied Behavior Analysis.

The "replication crisis" describes recent difficulties in replicating studies in various scientific fields, most notably psychology. The available evidence primarily documents replication failures for group research designs. However, we argue that contingencies of publication bias that led to the "replication crisis" also operate on applied behavior analysis (ABA) researchers who use single-case research designs (SCRD). This bias strongly favors publication of SCRD studies that show strong experimental effect, and disfavors publication of studies that show less robust effect. The resulting research literature may unjustifiably inflate confidence about intervention effects, limit researchers' ability to delineate intervention boundary conditions, and diminish the credibility of our science. To counter problems of publication bias in ABA, we recommend that journals that publish SCRD research establish journal standards for publication of noneffect studies; that our research community adopt open sharing of SCRD protocols and data; and that members of our community routinely publish systematic literature reviews that include gray (i.e., unpublished) research.

Effects of reinforcement without extinction on increasing compliance with nail cutting: A systematic replication.

Personal hygiene routines, such as nail cutting, are essential for maintaining good health. However, individuals with autism spectrum disorder (ASD) and other developmental disabilities often struggle to comply with essential, personal hygiene routines. We conducted a systematic replication of Schumacher and Rapp (2011), Shabani and Fisher (2006), and Bishop et al. (2013) to evaluate an intervention that did not require escape extinction for increasing compliance with nail cutting. With two adolescents diagnosed with ASD who resisted nail cutting, we evaluated the effects of delivering a preferred edible item contingent on compliance with nail cutting. Results indicated that the treatment reduced participants' escape responses and increased their compliance with nail cutting.

Meta-Analysis of Single-Case Research Design Studies on Instructional Pacing.

More than four decades of research on instructional pacing has yielded varying and, in some cases, conflicting findings. The purpose of this meta-analysis was to synthesize single-case research design (SCRD) studies on instructional pacing to determine the relative benefits of brisker or slower pacing. Participants were children and youth with and without disabilities in educational settings, excluding higher education. Tau-U, a non-parametric statistic for analyzing data in SCRD studies, was used to determine effect size estimates. The article extraction yielded 13 instructional pacing studies meeting contemporary standards for high quality SCRD research. Eleven of the 13 studies reported small to large magnitude effects when two or more pacing parameters were compared, suggesting that instructional pacing is a robust instructional variable. Brisker instructional pacing with brief inter-trial interval (ITI) produced small increases in correct responding and medium to large reductions in challenging behavior compared with extended ITI. Slower instructional pacing with extended wait-time produced small increases in correct responding, but also produced small increases in challenging behavior compared with brief wait-time. Neither brief ITI nor extended wait-time meets recently established thresholds for evidence-based practice, highlighting the need for further instructional pacing research.

An iPad™-based picture and video activity schedule increases community shopping skills of a young adult with autism spectrum disorder and intellectual disability.

OBJECTIVE: To evaluate the iPad 2™ with Book Creator™ software to provide visual cues and video prompting to teach shopping skills in the community to a young adult with an autism spectrum disorder and intellectual disability. METHODS: A multiple probe across settings design was used to assess effects of the intervention on the participant's independence with following a shopping list in a grocery store across three community locations. RESULTS: Visual cues and video prompting substantially increased the participant's shopping skills within two of the three community locations, skill increases maintained after the intervention was withdrawn, and shopping skills generalized to two untaught shopping items. Social validity surveys suggested that the participant's parent and staff favorably viewed the goals, procedures, and outcomes of intervention. CONCLUSIONS: The iPad 2™ with Book Creator™ software may be an effective way to teach independent shopping skills in the community; additional replications are needed.

Teaching self-control with qualitatively different reinforcers.

This study examined the effectiveness of using qualitatively different reinforcers to teach self-control to an adolescent boy who had been diagnosed with an intellectual disability. First, he was instructed to engage in an activity without programmed reinforcement. Next, he was instructed to engage in the activity under a two-choice fixed-duration schedule of reinforcement. Finally, he was exposed to self-control training, during which the delay to a more preferred reinforcer was initially short and then increased incrementally relative to the delay to a less preferred reinforcer. Self-control training effectively increased time on task to earn the delayed reinforcer.

Effects of social stories on prosocial behavior of preschool children with autism spectrum disorders.

Social Stories are a popular intervention for preschool children with autism spectrum disorders (ASD), but little research on Social Stories has been conducted with this population. This study investigated the effects of Social Stories on prosocial behavior of three preschool children with ASD in an inclusive setting. An ABAB design was used for two participants, while an ABACBC was used for the third. Social Stories increased appropriate behavior and decreased inappropriate behavior for two participants. The addition of verbal prompts (condition C) was necessary to increase appropriate behavior for the third participant. Maintenance probes were conducted to assess whether stories became imbedded in classroom routines. Results are discussed in relation to applications, study limitations, and areas for future research.

Multiply controlled verbal operants: an analysis and extension to the picture exchange communication system.

This paper presents Skinner's (1957) analysis of verbal behavior as a framework for understanding language acquisition in children with autism. We describe Skinner's analysis of pure and impure verbal operants and illustrate how this analysis may be applied to the design of communication training programs. The picture exchange communication system (PECS) is a training program influenced by Skinner's framework. We describe the training sequence associated with PECS and illustrate how this sequence may establish multiply controlled verbal behavior in children with autism. We conclude with an examination of how Skinner's framework may apply to other communication modalities and training strategies.