An Internal Consistency Reliability Study of the Catalyst Datafinch Applied Behavior Analysis Data Collection Application With Autistic Individuals.

Introduction Many psychometric studies have scrutinized the dependability of different instruments for evaluating and treating autism using applied behavior analysis (ABA). However, there has been no exploration into the psychometric attributes of the Catalyst Datafinch Applied Behavior Analysis Data Collection Application, namely, internal consistency reliability measures. Materials and methods  Four datasets were extracted (n=100, 98, 103, and 62) from published studies at The Oxford Center, Brighton, MI, ranging from March 19, 2023, through January 8, 2024, using Catalyst Datafinch as the data collection tool. All data were gathered by Board Certified Behavior Analysts (BCBAs) and behavioral technicians and designed to replicate how practitioners collect traditional paper and pencil data. SPSS Statistics (v. 29.0) computed internal consistency reliability measures, including Cronbach's alpha, inter-item, split-half, and interclass correlation coefficients. Results  Dataset #1: Cronbach's alpha was 0.916 with seven items, indicating excellent reliability. Cronbach's split-half reliability for Part 1 was 0.777, indicating good reliability, and for Part 2 was 0.972, indicating excellent reliability. Guttman split-half coefficient was 0.817, indicating good reliability. Inter-item correlation coefficients ranged from 0.474 to 0.970. The average measures interclass correlation (ICC) was 0.916, indicating excellent reliability. Single measures (ICC) reliability was 0.609, indicating acceptable reliability. Dataset #2: Cronbach's alpha was 0.954 with three items, indicating excellent reliability. Cronbach's split-half reliability for Part 1 was 0.912, indicating excellent reliability, and for Part 2 was 0.975, indicating excellent reliability. Guttman split-half coefficient was 0.917, indicating excellent reliability. Inter-item correlation coefficients ranged from 0.827 to 0.977. Average measures (ICC) was 0.954, indicating excellent reliability. Single measures (ICC) reliability was 0.875, indicating good reliability. Dataset #3: Cronbach's alpha was 0.974 with three items, indicating excellent reliability. Cronbach's split-half reliability for Part 1 was 0.978, indicating excellent reliability. Split-half reliability for Part 2 was 0.970, indicating excellent reliability. Guttman split-half coefficient was 0.935, indicating excellent reliability. Inter-item correlation coefficients ranged from 0.931 to 0.972. The average measures (ICC) was 0.974, indicating excellent reliability. Single measures (ICC) reliability was 0.926, indicating excellent reliability. Dataset #4: Cronbach's alpha was 0.980 with 12 items, indicating excellent reliability. Cronbach's split-half reliability for Part 1 was 0.973, indicating excellent reliability. Split-half reliability for Part 2 was 0.996, indicating excellent reliability. Guttman split-half coefficient was 0.838, indicating good reliability. Inter-item correlation coefficients ranged from 0.692 to 0.999. The average measures (ICC) was 0.980, indicating excellent reliability. Single measures (ICC) reliability was 0.804, indicating good reliability. Conclusions These results suggest that Catalyst Datafinch demonstrates high internal consistency reliability when used with individuals with autism. This indicates that the application is reliable for collecting and analyzing behavioral data in this population. The ratings ranged from good to excellent, indicating a high consistency in the measurements.

Replicative Study of the Impacts of Applied Behavior Analysis on Target Behaviors in Individuals With Autism Using Repeated Measures.

Background  The effectiveness of interventions based on applied behavior analysis (ABA) for individuals with autism has been well documented in numerous meta-analyses, systematic reviews, and cost-benefit analyses. However, an observed 'efficacy-effectiveness gap' exists, which can be attributed to various factors. This third replication study, therefore, has significant implications for the field. By assessing the impact of ABA treatment, specifically involving discrete trial training and mass trials, within a naturalistic environment, the study provides valuable insights that can inform and improve the delivery of ABA treatments in real-world settings. Methods  The study was conducted using a repeated measures research design. Retrospective chart review data were collected from 62 individuals with autism, age (M=8.65, SD=4.53), all of whom were level two autistic and required moderate support in communication, socialization, and daily life. These individuals received ABA treatment over five months. The study measured cumulative target behaviors using a repeated measures design, which allowed for the identification of statistically significant differences across 12 time points. This robust methodology ensures the validity and reliability of the study's findings. Results  Mixed repeated measures analysis of variance (ANOVA) indicated statistical significance (sphericity assumed), F(11,495) = 55.432, p < 0.001 (time). Multiple comparisons using bootstrapped paired t-tests showed p < 0.05 on time points 1-8 and non-significance (p > 0.05) on time points 9-12. There was a significant interaction effect (sphericity assumed) with time x (age category), F(44,495) = 2.338, p < 0.001. Interaction contrasts indicated statistically significant differences over time, mainly within the one-year to four-year-old, five to eight-year-old, and most in the nine to 12-year-old age groups. There was some significance within the 13- to 16-year-old age group and no significance within the 17- to 26-year-old age group. Conclusions  Over five months, individuals with autism who underwent ABA treatments demonstrated a statistically significant enhancement in general target behaviors. This finding is crucial as it underscores the effectiveness of ABA treatments in a naturalistic environment. Moreover, the study's discovery of a significant interaction between time and age in these behaviors provides valuable insights into the impact of age on treatment outcomes. Extensive large-N studies of general ABA broad effectiveness and repeated measures designs are lacking and can lead to further research to improve quality and outcomes. These findings contribute to the body of empirical evidence and emphasize the importance of replicative efficacy studies in ensuring the reliability of research findings.

The Effects of Applied Behavior Analysis on Verbal Behavior With Autistic Individuals Using the Verbal Behavior Milestones Assessment and Placement Program (VBMAPP) and the Assessment of Basic Language and Learning Skills (ABLLS).

Introduction  Applied behavior analysis (ABA) is a fundamental practice-based intervention for treating autism spectrum disorder (ASD). Few studies have directly measured and evaluated the effects of ABA on verbal behaviors, mainly using the Verbal Behavior Milestones Assessment and Placement Program (VBMAPP) and the Assessment of Basic Language and Learning Skills (ABLLS) as outcome measures. This study aims to fill this gap by examining the relationship between ABA interventions and the enhancement of verbal skills, as measured by the VBMAPP and the ABLLS, in a convenience sample of individuals with ASD.  Materials and methods At The Oxford Centers (TOCs) in Brighton and Troy, Michigan, USA, 33 individuals with autism received treatment from January 2018 to July 2021, spanning 43 months. A pretest-posttest design was employed to retrospectively examine any impacts between ABA interventions and alterations in verbal scores among individuals with ASD. Depending on developmental age, all subjects underwent two verbal assessments with a six-month interval in-between. Twelve children were administered the VBMAPP, while 21 were given the ABLLS. Results Paired t-tests for pretest and posttest VBMAPP subscales resulted in statistically significant effects (p<0.05) for (VBMAPP - Mand), (VBMAPP - Tact), (VBMAPP - Listener Responding), (VBMAPP - Visual Perceptual Skills and Matching-to-Sample), (VBMAPP -Independent Play), (VBMAPP - Social Play), (VBMAPP - Motor Imitation), (VBMAPP - Spontaneous Vocalization), (VBMAPP - Intraverbal), (VBMAPP - Group Behavior), and (VBMAPP - Linguistic Structure). As measured by Cohen's d, effect sizes were moderate to mostly high (-0.623 to -1.688). There were non-significant results (p>0.05) for (VBMAPP - Listener Responding by Feature, Function, and Class) and (VBMAPP - Echoic). Paired t-tests for pretest and posttest ABLLS subscales resulted in statistically significant effects (p<.05) for all ABLLS scales: (ABLLS - Receptive Language), (ABLLS - Requests), (ABLLS - Labeling), (ABLLS - Intraverbals), (ABLLS - Spontaneous Vocalizations), (ABLLS - Syntax Grammar), (ABLLS - Social Interactions), and (ABLLS - Generalized Responding). As measured by Cohen's d, effect sizes were moderate to mostly high (-0.656 to -1.372). Conclusions  The administration of ABA treatments had a noteworthy influence, with statistically significant impacts on improving verbal behaviors on 11 of the 13 VBMAPP scales and all of the ABLLS scales. As measured by Cohen's d, effect sizes were moderate to high for both scales. These findings underscore the importance and effectiveness of ABA interventions in enhancing verbal skills in children with ASD. However, it's crucial to note that further confirmatory studies are required to verify the reliability of these original findings, emphasizing the ongoing need for research in this field.

Examining the Effects of Discrete Trials, Mass Trials, and Naturalistic Environment Training on Autistic Individuals Using Repeated Measures.

INTRODUCTION: Behavioral interventions based on applied behavior analysis (ABA) form current evidence-based practices in treating autism spectrum disorder (ASD). Research is scarce relative to the broad effects of intensive repetitive, discrete trial training, and mass trials combined with a naturalistic environment as measured by overall general target behaviors. The primary objective of this study was to evaluate the effectiveness of a mixed behavioral model consisting of discrete trial training and mass trial interventions in the naturalistic environment, using a repeated measures design with a retrospective snapshot cohort of 93 individuals with autism. METHODS: A repeated measures analysis tracked 89 autistic children with four adult autistic individuals over seven time points during a three-month snapshot period from March 19, 2023, to June 11, 2023. This study determined the effectiveness of applied behavior analysis (ABA) interventions combining discrete trial training, mass trials, and naturalistic environment training on mastered broad target behaviors in autistic individuals using a mixed (between and within) ANOVA statistical design. RESULTS: Mixed (between and within) ANOVA indicated overall statistical significance, F (6,674)=45.447, p<0.001, partial eta squared=0.365 across time. These results indicated a large effect size. Multiple comparisons showed statistical significance (p<0.001) on all 21 multiple comparisons between timepoints. There was also a significant interaction effect with time × age category, F (24,474)=2.961, p<0.001, partial eta squared=0.130. These results also indicated a large effect size. CONCLUSIONS: Autistic individuals who received applied behavior analysis combining discrete trial training, mass trials, and naturalistic environment training intervention demonstrated statistically significant improvement in target behaviors over the three-month snapshot period, the most prominent being in the 13-16 years age category.

Impact of Applied Behavior Analysis on Autistic Children Target Behaviors: A Replication Using Repeated Measures.

INTRODUCTION: Applied behavior analysis (ABA) is a primary evidence-based practice in treating autism spectrum disorder (ASD). Ongoing research is needed to report the results of ABA relative to attaining target behaviors. This study aims to replicate the results of previous research to determine the effectiveness of ABA of target behaviors in autistic children with a new timepoint sample of data.  Materials & methods: A repeated measures analysis tracked 98 autistic children, which included four adult participants, over three timepoints during a one-month snapshot period from 6/7/23 to 7/7/23. This study used a retrospective chart review to gather data on target behaviors to determine the effectiveness of ABA treatments across age categories. A mixed (between x within) analysis of variance (ANOVA) and subsequent post hoc and interaction contrasts were used to determine statistical significance. RESULTS: Mixed (between x within) ANOVA indicated statistical significance (sphericity assumed), F(2,160) = 32.893, and p < 0.05, across time. Using bootstrapped paired t-tests, multiple comparisons indicated p < 0.001 on all three multiple comparisons, with Bonferroni corrected α = 0.017. There was also a non-significant interaction effect (sphericity assumed) with (time) x (age category), F(8,160) = 0.333, p = 0.952, likely due to sizeable within-group variation resulting in a lowered statistical power.  Conclusions: This replication found that autistic children receiving the ABA intervention demonstrated statistically significant improvement in target behaviors over the one-month snapshot period.

Treating Target Behaviors of Autistic Individuals With Applied Behavior Analysis: An Ongoing Replication Study.

Background Ongoing outcome data replication on target behaviors with autistic individuals using applied behavior analysis (ABA) confirms its effectiveness and remains an essential evidenced-based standard of care. This replication study aims to further confirm the impact of discrete trial training and mass trials on general target behaviors within a naturalistic environment. Methods Data was gathered from 92 children and four adult autistic individuals over one month from 7/7/23 to 8/8/23 using a repeated measures design. This study used a retrospective chart review with general target behaviors to determine the effectiveness of ABA treatments using discrete trial training and mass trials across time and age categories in a naturalistic environment. Results A mixed analysis of variance (ANOVA) indicated statistical significance (sphericity assumed), F(2,168) = 31.663, p < 0.001 (time). Multiple comparisons using bootstrapped paired t-tests indicated p < 0.001 on the three comparisons. There was a significant interaction effect (sphericity assumed) with time x age category, F(8,168) = 2.918, p = 0.004. Interaction contrasts indicated statistically significant differences over time within the 1-4 years, 5-8 years, and a portion of 9-12 years, and not within the 13-16 years and 17-73 years age groups. Conclusions Autistic individuals receiving ABA demonstrated statistically significant improvement in target behaviors over one month. There was a significant interaction between time and age on target behaviors, suggesting a significant association between time and age categories. The reporting of ongoing intervention outcomes provides further justification for continued treatments relative to target behavior mastery with autistic individuals.

Predicting Behavioral Target Mastery With Age, Intensity and Duration, Open Targets, and Maintenance Failure With Applied Behavior Analysis in Individuals With Autism.

Background Researchers have studied the effects of age, treatment intensity, and treatment duration with applied behavior analysis (ABA) outcomes in autistic individuals. This study's primary objective was to evaluate the predictive capabilities of age, intensity, and duration of treatment, open behavioral targets, and behavior maintenance failure on behavioral target mastery. Methods A retrospective cohort of 100 autistic individuals treated with ABA with functional analysis and discrete trial training, mass trials, and naturalistic training were treated and analyzed. Target behavioral mastery data was collected via a retrospective chart review contained within the "Catalyst" tracking software. ABA treatment was administered for three months between March 19, 2023, and June 11, 2023. Multiple linear regression was performed using the percentage of behavioral targets mastered as the dependent variable. The independent variables were age, average trials to behavioral mastery, average teaching days to behavioral mastery, and percentage of behavioral targets that failed in maintenance. Results The multiple linear regression model was statistically significant (R=0.443, R²=0.196, adjusted R2=0.150, F(5, 87)=4.239, p=0.002). The average teaching days to mastery (ß=0.416, p=0.019) and percentage of targets failed in maintenance (ß=0.201, p=0.047) significantly predicted the percentage of behavioral targets mastered. However, age (ß=0.079, p=0.419), average trials to mastery (ß=-0.271, p=0.114), and open targets (ß=0.184, p=0.081) did not significantly predict the percentage of behavioral targets mastered. The analysis of variance (ANOVA) resulted in non-significant (p>0.05) age group differences between the above variables. Conclusions The predictor variables average teaching days to mastery (intensity and duration) and percentage of targets failed in maintenance had a statistically significant effect on the percentage of behavioral targets mastered. The predictor variables age, average trials to behavioral mastery (intensity and duration), and open behavioral targets had a non-significant influence on the percentage of behavioral targets mastered. A non-significant difference between age groups was found in all study variables.