Assessing Growth of BACB Certificants (1999-2019).

Geographic distribution patterns of board certified behavior analysts may be useful in analyzing the growth of the field. First, we present an international snapshot of Behavior Analyst Certification Board (BACB) certificants, then analyze relative growth rates between countries from 1999 to 2019. This is followed by an in depth review of certificant distribution patterns in the United States and Canada, as well as the ratios of experienced behavior analysts to new certificants. These data highlight regions with a potential deficit of qualified supervisors. There are factors that influence different dispersal patterns, and without drilling deeper into the data we may be unable to effectively identify or influence them in order meet the specific needs of a geographic region. Supplementary Information: The online version contains supplementary material available at 10.1007/s40614-023-00370-5.

The Family Game to support parents with intellectual disability in managing challenging behaviours: A replication.

BACKGROUND: Although many parents with intellectual disability (ID) demonstrate good parenting practices, some parents experience difficulties in managing challenging behaviours. One potential solution to this issue involves using The Family Game, a program designed to teach parents with ID how to manage challenging behaviours in their child. AIMS: The purpose of our study was to conduct an independent replication of an investigation that had been performed by the developer of the program. MATERIALS & METHODS: We used a multiple baseline design to examine the effects of The Family Game on the behaviour of two parents with ID who had a 3-year-old child. RESULTS: Similarly to the original study, our results indicate that The Family Game improved the use of effective parenting strategies during role play, but that these gains failed to generalise to real-life settings. CONCLUSION: The study further supports the necessity of adding novel strategies to the game to better promote generalisation.

Brief Report: Virtual Reality to Raise Awareness About Autism.

PURPOSE: The purpose of the study was to develop and test a virtual reality application designed to put the participants "in the shoes" of an autistic person during a routine task. METHOD: The study involved a randomized controlled trial that included 103 participants recruited from a technical college. Each participant responded to three questionnaires to measure attitudes, knowledge, and openness toward autism. Prior to responding to these questionnaires, the participants in the experimental group also completed an 8-min virtual reality simulation designed by the research team in collaboration with autistic individuals. RESULTS: The participants who completed the virtual reality simulation reported better attitudes, more knowledge, and higher openness toward autism than the participants in the control group. CONCLUSION: The results of the study suggest that virtual reality simulations are promising tools to raise awareness about autism.

Screening for Psychological Distress in Healthcare Workers Using Machine Learning: A Proof of Concept.

The purpose of this study was to train and test preliminary models using two machine learning algorithms to identify healthcare workers at risk of developing anxiety, depression, and post-traumatic stress disorder. The study included data from a prospective cohort study of 816 healthcare workers collected using a mobile application during the first two waves of COVID-19. Each week, the participants responded to 11 questions and completed three screening questionnaires (one for anxiety, one for depression, and one for post-traumatic stress disorder). Then, the research team selected two questions (out of the 11), which were used with biological sex to identify whether scores on each screening questionnaire would be positive or negative. The analyses involved a fivefold cross-validation to test the accuracy of models based on logistic regression and support vector machines using cross-sectional and cumulative measures. The findings indicated that the models derived from the two questions and biological sex accurately identified screening scores for anxiety, depression, and post-traumatic stress disorders in 70% to 80% of cases. However, the positive predictive value never exceeded 50%, underlining the importance of collecting more data to train better models. Our proof of concept demonstrates the feasibility of using machine learning to develop novel models to screen for psychological distress in at-risk healthcare workers. Developing models with fewer questions may reduce burdens of active monitoring in practical settings by decreasing the weekly assessment duration.

Waiting for baseline stability in single-case designs: Is it worth the time and effort?

Researchers and practitioners often use single-case designs (SCDs), or n-of-1 trials, to develop and validate novel treatments. Standards and guidelines have been published to provide guidance as to how to implement SCDs, but many of their recommendations are not derived from the research literature. For example, one of these recommendations suggests that researchers and practitioners should wait for baseline stability prior to introducing an independent variable. However, this recommendation is not strongly supported by empirical evidence. To address this issue, we used Monte Carlo simulations to generate graphs with fixed, response-guided, and random baseline lengths while manipulating trend and variability. Then, our analyses compared the type I error rate and power produced by two methods of analysis: the conservative dual-criteria method (a structured visual aid) and a support vector classifier (a model derived from machine learning). The conservative dual-criteria method produced fewer errors when using response-guided decision-making (i.e., waiting for stability) and random baseline lengths. In contrast, waiting for stability did not reduce decision-making errors with the support vector classifier. Our findings question the necessity of waiting for baseline stability when using SCDs with machine learning, but the study must be replicated with other designs and graph parameters that change over time to support our results.

Using AB Designs With Nonoverlap Effect Size Measures to Support Clinical Decision-Making: A Monte Carlo Validation.

Single-case experimental designs often require extended baselines or the withdrawal of treatment, which may not be feasible or ethical in some practical settings. The quasi-experimental AB design is a potential alternative, but more research is needed on its validity. The purpose of our study was to examine the validity of using nonoverlap measures of effect size to detect changes in AB designs using simulated data. In our analyses, we determined thresholds for three effect size measures beyond which the type I error rate would remain below 0.05 and then examined whether using these thresholds would provide sufficient power. Overall, our analyses show that some effect size measures may provide adequate control over type I error rate and sufficient power when analyzing data from AB designs. In sum, our results suggest that practitioners may use quasi-experimental AB designs in combination with effect size to rigorously assess progress in practice.

Changes in Autistic Symptoms and Adaptive Functioning of Children Receiving Early Behavioral Intervention in a Community Setting: A Latent Growth Curve Analysis.

Despite showing effects in well-controlled studies, the extent to which early intensive behavioral intervention (EBI) produces positive changes in community-based settings remains uncertain. Thus, our study examined changes in autistic symptoms and adaptive functioning in 233 children with autism receiving EBI in a community setting. The results revealed nonlinear changes in adaptive functioning characterized by significant improvements during the intervention and a small linear decrease in autistic symptoms from baseline to follow-up. The intensity of intervention, initial age, IQ and autistic symptoms were associated either with progress during the intervention or maintenance during the follow-up. The next step to extend this line of research involves collecting detailed data about intervention strategies and implementation fidelity to produce concrete recommendations for practitioners.

Tutorial: Artificial neural networks to analyze single-case experimental designs.

Since the start of the 21st century, few advances have had as far-reaching impact in science as the widespread adoption of artificial neural networks in fields as diverse as fundamental physics, clinical medicine, and psychology. In research methods, one promising area for the adoption of artificial neural networks involves the analysis of single-case experimental designs. Given that these types of networks are not generally part of training in the psychological sciences, the purpose of our article is to provide a step-by-step introduction to using artificial neural networks to analyze single-case designs. To this end, we trained a new model using data from a Monte Carlo simulation to analyze multiple baseline graphs and compared its outcomes with traditional methods of analysis. In addition to showing that artificial neural networks may produce less error than other methods, this tutorial provides information to facilitate the replication and extension of this line of work to other designs and datasets. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Brief Report: Machine Learning for Estimating Prognosis of Children with Autism Receiving Early Behavioral Intervention-A Proof of Concept.

Although early behavioral intervention is considered as empirically-supported for children with autism, estimating treatment prognosis is a challenge for practitioners. One potential solution is to use machine learning to guide the prediction of the response to intervention. Thus, our study compared five machine algorithms in estimating treatment prognosis on two outcomes (i.e., adaptive functioning and autistic symptoms) in children with autism receiving early behavioral intervention in a community setting. Each machine learning algorithm produced better predictions than random sampling on both outcomes. Those results indicate that machine learning is a promising approach to estimating prognosis in children with autism, but studies comparing these predictions with those produced by qualified practitioners remain necessary.

Some Characteristics and Arguments in Favor of a Science of Machine Behavior Analysis.

Researchers and practitioners recognize four domains of behavior analysis: radical behaviorism, the experimental analysis of behavior, applied behavior analysis, and the practice of behavior analysis. Given the omnipresence of technology in every sphere of our lives, the purpose of this conceptual article is to describe and argue in favor of a fifth domain: machine behavior analysis. Machine behavior analysis is a science that examines how machines interact with and produce relevant changes in their external environment by relying on replicability, behavioral terminology, and the philosophical assumptions of behavior analysis (e.g., selectionism, determinism, parsimony) to study artificial behavior. Arguments in favor of a science of machine behavior include the omnipresence and impact of machines on human behavior, the inability of engineering alone to explain and control machine behavior, and the need to organize a verbal community of scientists around this common issue. Regardless of whether behavior analysts agree or disagree with this proposal, I argue that the field needs a debate on the topic. As such, the current article aims to encourage and contribute to this debate.

Agreement between visual inspection and objective analysis methods: A replication and extension.

Behavior analysts typically rely on visual inspection of single-case experimental designs to make treatment decisions. However, visual inspection is subjective, which has led to the development of supplemental objective methods such as the conservative dual-criteria method. To replicate and extend a study conducted by Wolfe et al. (2018) on the topic, we examined agreement between the visual inspection of five raters, the conservative dual-criteria method, and a machine-learning algorithm (i.e., the support vector classifier) on 198 AB graphs extracted from clinical data. The results indicated that average agreement between the 3 methods was generally consistent. Mean interrater agreement was 84%, whereas raters agreed with the conservative dual-criteria method and the support vector classifier on 84% and 85% of graphs, respectively. Our results indicate that both objective methods produce results consistent with visual inspection, which may support their future use.

Machine Learning to Support Visual Inspection of Data: A Clinical Application.

Practitioners in pediatric feeding programs often rely on single-case experimental designs and visual inspection to make treatment decisions (e.g., whether to change or keep a treatment in place). However, researchers have shown that this practice remains subjective, and there is no consensus yet on the best approach to support visual inspection results. To address this issue, we present the first application of a pediatric feeding treatment evaluation using machine learning to analyze treatment effects. A 5-year-old male with autism spectrum disorder participated in a 2-week home-based, behavior-analytic treatment program. We compared interrater agreement between machine learning and expert visual analysts on the effects of a pediatric feeding treatment within a modified reversal design. Both the visual analyst and the machine learning model generally agreed about the effectiveness of the treatment while overall agreement remained high. Overall, the results suggest that machine learning may provide additional support for the analysis of single-case experimental designs implemented in pediatric feeding treatment evaluations.

Machine learning to analyze single-case graphs: A comparison to visual inspection.

Behavior analysts commonly use visual inspection to analyze single-case graphs, but studies on its reliability have produced mixed results. To examine this issue, we compared the Type I error rate and power of visual inspection with a novel approach-machine learning. Five expert visual raters analyzed 1,024 simulated AB graphs, which differed on number of points per phase, autocorrelation, trend, variability, and effect size. The ratings were compared to those obtained by the conservative dual-criteria method and two models derived from machine learning. On average, visual raters agreed with each other on only 75% of graphs. In contrast, both models derived from machine learning showed the best balance between Type I error rate and power while producing more consistent results across different graph characteristics. The results suggest that machine learning may support researchers and practitioners in making fewer errors when analyzing single-case graphs, but replications remain necessary.

Correction to: Machine Learning to Analyze Single-Case Data: A Proof of Concept.

[This corrects the article DOI: 10.1007/s40614-020-00244-0.].

Machine Learning for Supplementing Behavioral Assessment.

The Questions About Behavioral Function (QABF) has a high degree of convergent validity, but there is still a lack of agreement between the results of the assessment and the results of experimental function analysis. Machine learning (ML) may improve the validity of assessments by using data to build a mathematical model for more accurate predictions. We used published QABF and subsequent functional analyses to train ML models to identify the function of behavior. With ML models, predictions can be made from indirect assessment results based on learning from results of past experimental functional analyses. In Experiment 1, we compared the results of five algorithms to the QABF criteria using a leave-one-out cross-validation approach. All five outperformed the QABF assessment on multilabel accuracy (i.e., percentage of predictions with the presence or absence of each function indicated correctly), but false negatives remained an issue. In Experiment 2, we augmented the data with 1,000 artificial samples to train and test an artificial neural network. The artificial network outperformed other models on all measures of accuracy. The results indicated that ML could be used to inform conditions that should be present in a functional analysis. Therefore, this study represents a proof-of-concept for the application of machine learning to functional assessment.

Effects of an Interactive Web Training to Support Parents in Reducing Challenging Behaviors in Children with Autism.

Many children with autism spectrum disorder (ASD) engage in challenging behaviors, which may interfere with their daily functioning, development, and well-being. To address this issue, we conducted a four-week randomized waitlist control trial to examine the effects of a fully self-guided interactive web training (IWT) on (a) child engagement in challenging behaviors and (b) parental intervention. After 4 weeks, parents in the treatment group reported lower levels of challenging behaviors in their children and more frequent use of behavioral interventions than those in the waitlist groups. Furthermore, within-group analyses suggest that these changes persisted up to 12 weeks following completion of the IWT. Our results highlight the potential utility of web training, but our high attrition rate and potential side effects prevent us from recommending the training as a standalone treatment.

Brief Report: Mobile Technology to Support Parents in Reducing Stereotypy.

Although behavioral interventions have been known to effectively reduce stereotypy in children with ASD, these types of interventions are not accessible to all families. In response to this issue, we evaluated the effects of the iSTIM, an iOS application designed to support parents in the reduction of stereotypy in their child with ASD. We used a series of AB designs to determine the effectiveness of the iSTIM on stereotypy using parents as behavior change agents. The use of iSTIM by the parents led to a reduction in stereotypy for six of seven participants. Our results suggest that the use of technology may be a cost effective and easily accessible method for parents to reduce stereotypy in their child with ASD.

Tutorial: Applying Machine Learning in Behavioral Research.

Machine-learning algorithms hold promise for revolutionizing how educators and clinicians make decisions. However, researchers in behavior analysis have been slow to adopt this methodology to further develop their understanding of human behavior and improve the application of the science to problems of applied significance. One potential explanation for the scarcity of research is that machine learning is not typically taught as part of training programs in behavior analysis. This tutorial aims to address this barrier by promoting increased research using machine learning in behavior analysis. We present how to apply the random forest, support vector machine, stochastic gradient descent, and k-nearest neighbors algorithms on a small dataset to better identify parents of children with autism who would benefit from a behavior analytic interactive web training. These step-by-step applications should allow researchers to implement machine-learning algorithms with novel research questions and datasets.

Artificial intelligence for the measurement of vocal stereotypy.

Both researchers and practitioners often rely on direct observation to measure and monitor behavior. When these behaviors are too complex or numerous to be measured in vivo, relying on direct observation using human observers increases the amount of resources required to conduct research and to monitor the effects of interventions in practice. To address this issue, we conducted a proof of concept examining whether artificial intelligence could measure vocal stereotypy in individuals with autism. More specifically, we used an artificial neural network with over 1,500 minutes of audio data from 8 different individuals to train and test models to measure vocal stereotypy. Our results showed that the artificial neural network performed adequately (i.e., session-by-session correlation near or above .80 with a human observer) in measuring engagement in vocal stereotypy for 6 of 8 participants. Additional research is needed to further improve the generalizability of the approach.

How Many Tiers Do We Need? Type I Errors and Power in Multiple Baseline Designs.

Design quality guidelines typically recommend that multiple baseline designs include at least three demonstrations of effects. Despite its widespread adoption, this recommendation does not appear grounded in empirical evidence. The main purpose of our study was to address this issue by assessing Type I error rate and power in multiple baseline designs. First, we generated 10,000 multiple baseline graphs, applied the dual-criteria method to each tier, and computed Type I error rate and power for different number of tiers showing a clear change. Second, two raters categorized the tiers for 300 multiple baseline graphs to replicate our analyses using visual inspection. When multiple baseline designs had at least three tiers and two or more of these tiers showed a clear change, the Type I error rate remained adequate (< .05) while power also reached acceptable levels (> .80). In contrast, requiring all tiers to show a clear change resulted in overly stringent conclusions (i.e., unacceptably low power). Therefore, our results suggest that researchers and practitioners should carefully consider limitations in power when requiring all tiers of a multiple baseline design to show a clear change in their analyses.