Intervention selection profile-function: An examination of decisional accuracy relative to traditional FBA data.

The purpose of this study was to examine the accuracy of function-based decisions made in consideration of scores from the Intervention Selection Profile-Function (ISP-Function), a tool founded upon direct behavior rating (DBR) methodology. The ISP-Function is designed to be a brief measure, given the need for efficient and low-resource assessments in schools. Data from a previous investigation were used to create data reports for each of 34 elementary students with a history of exhibiting disruptive behavior in the classroom. The first report summarized ISP-Function data that the student's classroom teacher collected. The second report was representative of more typical functional behavior assessment (FBA), summarizing data collected via a functional assessment interview with the teacher, as well as systematic direct observation data. Nine school psychologists conducted blind reviews of these reports and derived decisions regarding the function of each student's behavior (e.g., adult attention or escape/avoidance). Gwet's agreement coefficients were statistically significant and suggested fair to almost perfect correspondence between ISP-Function and FBA reports. Limitations and implications for practice are discussed herein. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Test anxiety, anxiety disorders, and school-related wellbeing: Manifestations of the same or different constructs?

Previous studies have shown that highly test anxious persons are more likely to meet criteria for an anxiety disorder and report more frequent symptoms of anxiety disorders than their low test anxious counterparts. However, it is unclear whether test anxiety should be treated as distinct to, or a manifestation of, anxiety disorders. Furthermore, the Dual Factor Model of Mental Health proposes that high subjective wellbeing cannot be solely inferred from the absence of psychopathology. To date, no studies have examined the Dual Factor Model in relation to test anxiety. In the present study, we examined how test anxiety, two common anxiety disorders (i.e., generalized anxiety disorder [GAD] and panic disorder [PD]), and subjective wellbeing in the school domain (i.e., school-related wellbeing) were related in a sample of 918 adolescents (M age = 15.77 years) using network analysis and latent profile analysis. Results from the network analysis indicated that test anxiety, GAD, PD, and school-related wellbeing were represented as distinct constructs. Bridge nodes were identified that linked test anxiety with GAD, PD, and school-related wellbeing. The latent profile analysis identified three of the four profiles predicted by the Dual Factor Model, including (a) troubled (i.e., low school-related wellbeing, high test anxiety, GAD, and PD), (b) complete mental health (i.e., high school-related wellbeing, low test anxiety, GAD, and PD), and (c) symptomatic but content (i.e., average school-related wellbeing, test anxiety, GAD, and PD). We concluded that test anxiety was distinct from, rather than a manifestation of, GAD and PD. We found support for the Dual Factor Model, albeit not unequivocal, using test anxiety as an additional indicator of psychopathology to that of GAD and PD.

Construct validity of the BASC-3 teacher rating scales: Independent hierarchical exploratory factor analyses with the normative sample.

The Behavior Assessment System for Children-Third Edition (BASC-3) is the most recent edition and the Teacher Rating Scales (TRS) was reported to be the most frequently used test in school psychology practice. Despite its popularity, there is a lack of independent empirical research regarding psychometric properties. The BASC-3 Manual, while quite detailed in many respects, lacks important details in reporting TRS item- and scale-level factor analyses limiting confidence in construct validity based on internal structure. The present study examined the latent factor structure of the BASC-3 TRS Preschool, Child, and Adolescent Clinical and Adaptive scales using best practices in exploratory factor analysis (EFA). EFA was conducted with the Clinical and Adaptive scales jointly, and with the Clinical scales separately, to aid interpretive clarity. Results indicated theoretically consistent alignment of the BASC-3 TRS Clinical scales to their specified factors (Externalizing, Internalizing, and School Problems) and an additional factor (Social Disengagement) was identified, suggesting a possible new latent construct for a composite scale score containing the Withdrawal and Atypicality scales. Variance partitioning applied to second-order EFA and model-based validity statistics, however, indicated that the composite scales (Externalizing, Internalizing, School Problems, and Social Disengagement) appear to lack sufficient unique variance for confident clinical interpretation in isolation. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Cognitive-behavioral intervention for test anxiety in adolescent students: do benefits extend to school-related wellbeing and clinical anxiety.

Background and objectives: Cognitive-behavioral interventions have been shown to be effective treatments for test anxiety. Studies on school-aged populations, however, are lacking. Design and methods: In the present study we evaluated a six-session cognitive-behavioral intervention for test anxiety in a sample of secondary school students aged 14-16 years preparing for high-stakes examinations. Furthermore, we extended outcomes to include school-related wellbeing and clinical anxiety. A screening procedure was used to identify highly test anxious persons who were randomly allocated to intervention or wait-list control groups. Results: Test anxiety showed a large reduction following intervention compared to control group participants who showed a moderate reduction. Clinical anxiety showed a small to moderate reduction following intervention compared to control group participants who showed a negligible reduction. The reduction in clinical anxiety was mediated by concurrent reductions in test anxiety. Conclusion: This supports an integrative network approach that deactivating core aspects of anxiety can deactivate associated networks of anxiety symptoms. The intervention showed no impact on school-related wellbeing which increased at a similar rate for both intervention and control group participants. This is likely because test anxiety is just one contributor of many to school-related wellbeing. Implications for school-based treatments are discussed.

Development and validation of the Intervention Skills Profile-Skills: A brief measure of student social-emotional and academic enabling skills.

The purpose of this study was to support the development and initial validation of the Intervention Selection Profile (ISP)-Skills, a brief 14-item teacher rating scale intended to inform the selection and delivery of instructional interventions at Tier 2. Teacher participants (n = 196) rated five students from their classroom across four measures (total student n = 877). These measures included the ISP-Skills and three criterion tools: Social Skills Improvement System (SSIS), Devereux Student Strengths Assessment (DESSA), and Academic Competence Evaluation Scales (ACES). Diagnostic classification modeling (DCM) suggested an expert-created Q-matrix, which specified relations between ISP-Skills items and hypothesized latent attributes, provided good fit to item data. DCM also indicated ISP-Skills items functioned as intended, with the magnitude of item ratings corresponding to the model-implied probability of attribute mastery. DCM was then used to generate skill profiles for each student, which included scores representing the probability of students mastering each of eight skills. Correlational analyses revealed large convergent relations between ISP-Skills probability scores and theoretically-aligned subscales from the criterion measures. Discriminant validity was not supported, as ISP-Skills scores were also highly related to all other criterion subscales. Receiver operating characteristic (ROC) curve analyses informed the selection of cut scores from each ISP-Skills scale. Review of classification accuracy statistics associated with these cut scores (e.g., sensitivity and specificity) suggested they reliably differentiated students with below average, average, and above average skills. Implications for practice and directions for future research are discussed, including those related to the examination of ISP-Skills treatment utility.

Reliability, validity, and accuracy of the intervention selection profile-Function: A brief functional assessment tool.

The purpose of this study was to evaluate the reliability, validity, and accuracy of scores from the Intervention Selection Profile-Function (ISP-Function): a brief functional assessment tool founded upon Direct Behavior Rating (DBR) methodology. Participants included 34 teacher-student dyads. Using the ISP-Function, teachers rated the extent to which students exhibited disruptive behavior, as well as the frequency with which disruptions were met with four consequences. Ratings were completed across three 10-min sessions, during which a research assistant also collected systematic direct observation (SDO) data regarding the same behavior and consequences. Results indicated adequate temporal reliability (≥.70) was attained for the adult attention and peer attention targets across the three ratings; in contrast, up to 8-18 data points would be needed to achieve adequate reliability across the remaining targets. Findings further suggested that while ISP-Function ratings of disruptive behavior, adult attention, and peer attention were moderately to highly correlated with SDO data, correlations were in the low range for the access to items/activities and escape/avoidance targets. Finally, analysis of difference scores showed that on average, mean ISP-Function scores fell within only 0.33 to 1.81 points of mean SDO scores (on the 0-10 DBR scale). Agreement coefficients indicative of exact score agreement were less consistent, suggesting accuracy ranged from poor to substantial. Results are promising, but future research is necessary to support applied ISP-Function use. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Co-occurrence of academic and behavioral risk within elementary schools: Implications for universal screening practices.

The purposes of this study were twofold. The first was to use latent class analysis to identify groupings of students defined by the presence or absence of academic or behavioral risk. The second was to determine whether these groups differed across various dichotomous academic and behavioral outcomes (e.g., suspensions, office discipline referrals, statewide achievement test failure). Students (N = 1,488) were sampled from Grades 3-5. All students were screened for academic risk using AIMSweb Reading Curriculum-Based Measure and AIMSweb Mathematics Computation, and behavioral risk using the Social, Academic, and Emotional Behavior Risk Screener (SAEBRS). Latent class analyses supported the fit of a three-class model, with resulting student classes defined as low-risk academic and behavior (Class 1), at-risk academic and high-risk behavior (Class 2), and at-risk math and behavior (Class 3). Logistic regression analyses indicated the classes demonstrated statistically significant differences statewide achievement scores, as well as suspensions. Further analysis indicated that the odds of all considered negative outcomes were higher for both groups characterized by risk (i.e., Classes 2 and 3). Negative outcomes were particularly likely for Class 2, with the odds of negative behavioral and academic outcomes being 6-15 and 112-169 times more likely, respectively. Results were taken to support an integrated approach to universal screening in schools, defined by the evaluation of both academic and behavioral risk. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Evidence for the interpretation of Social, Academic, and Emotional Behavior Risk Screener (SAEBRS) scores: An argument-based approach to screener validation.

In accordance with an argument-based approach to validation, the purpose of the current study was to yield evidence relating to Social, Academic, and Emotional Behavior Risk Screener (SAEBRS) score interpretation. Bifactor item response theory analyses were performed to examine SAEBRS item functioning. Structural equation modeling (SEM) was used to simultaneously evaluate intra- and inter-scale relationships, expressed through (a) a measurement model specifying a bifactor structure to SAEBRS items, and (b) a structural model specifying convergent and discriminant relations with an outcome measure (i.e., Behavioral and Emotional Screening System [BESS]). Finally, hierarchical omega coefficients were calculated in evaluating the model-based internal reliability of each SAEBRS scale. IRT analyses supported the adequate fit of the bifactor model, indicating items adequately discriminated moderate and high-risk students. SEM results further supported the fit of the latent bifactor measurement model, yielding superior fit relative to alternative models (i.e., unidimensional and correlated factors). SEM analyses also indicated the latent SAEBRS-Total Behavior factor was a statistically significant predictor of all BESS subscales, the SAEBRS-Academic Behavior predicted BESS Adaptive Skills subscales, and the SAEBRS-Emotional Behavior predicted the BESS Internalizing Problems subscale. Hierarchical omega coefficients indicated the SAEBRS-Total Behavior factor was associated with adequate reliability. In contrast, after accounting for the total scale, each of the SAEBRS subscales was associated with somewhat limited reliability, suggesting variability in these scores is largely driven by the Total Behavior scale. Implications for practice and future research are discussed.

Diagnostic accuracy of a universal screening multiple gating procedure: A replication study.

The purpose of this diagnostic accuracy study was to evaluate the sensitivity and specificity (among other indicators) of three universal screening approaches, including the Social, Academic, and Emotional Behavior Risk Screener (SAEBRS), a SAEBRS-based teacher nomination tool, and a multiple gating procedure (MGP). Each screening approach was compared to the BASC-2 Behavioral and Emotional Screening System (BESS), which served as a criterion indicator of student social-emotional and behavioral risk. All data were collected in a concurrent fashion. Participants included 704 students (47.7% female) from four elementary schools within the Midwestern United States (21.6% were at risk per the BESS). Findings yielded support for the SAEBRS, with sensitivity = .93 (95% confidence interval [.89-.97]), specificity = .91 (.89-.93), and correct classification = .92. Findings further supported the MGP, which yielded sensitivity = .81 (.74-.87), specificity = .93 (.91-.95), and correct classification = .91. In contrast, the teacher nomination tool yielded questionable levels of diagnostic accuracy (sensitivity = .86 [.80-.91], specificity = .74 [.70-.78], and correct classification = .76). Overall, findings were particularly supportive of SAEBRS diagnostic accuracy, suggesting the MGP might also serve as an acceptable approach to universal screening. Other implications for practice and directions for future research are discussed. (PsycINFO Database Record (c) 2018 APA, all rights reserved).

Screening for behavioral risk: Identification of high risk cut scores within the Social, Academic, and Emotional Behavior Risk Screener (SAEBRS).

The purpose of this study was to support the identification of Social, Academic, and Emotional Behavior Risk Screener (SAEBRS) cut scores that could be used to detect high-risk students. Teachers rated students across two time points (Time 1 n = 1,242 students; Time 2 n = 704) using the SAEBRS and the Behavioral and Emotional Screening System (BESS), the latter of which served as the criterion measure. Exploratory receiver operating characteristic (ROC) curve analyses of Time 1 data detected cut scores evidencing optimal levels of specificity and borderline-to-optimal levels of sensitivity. Cross-validation analyses of Time 2 data confirmed the performance of these cut scores, with all but one scale evidencing similar performance. Findings are considered particularly promising for the SAEBRS Total Behavior scale in detecting high-risk students. (PsycINFO Database Record

Use of direct behavior ratings to collect functional assessment data.

The purpose of this investigation was to evaluate the utility of Direct Behavior Rating Single Item Scale (DBR-SIS) methodology in collecting functional behavior assessment data. Specific questions of interest pertained to the evaluation of the accuracy of brief DBR-SIS ratings of behavioral consequences and determination of the type of training necessary to support such accuracy. Undergraduate student participants (N = 213; 62.0% male; 62.4% White) viewed video clips of students in a classroom setting, and then rated both disruptive behavior and 4 consequences of that behavior (i.e., adult attention, peer attention, escape/avoidance, and access to tangibles/activities). Results indicated training with performance feedback was necessary to support the generation of accurate disruptive behavior and consequence ratings. Participants receiving such support outperformed students in training-only, pretest-posttest, and posttest-only groups for disruptive behavior and all 4 DBR-SIS consequence targets. Future directions for research and implications for practice are discussed, including how teacher ratings may be collected along with other forms of assessment (e.g., progress monitoring) within an efficient Tier 2 assessment model. (PsycINFO Database Record

Psychometric defensibility of the Social, Academic, and Emotional Behavior Risk Screener (SAEBRS) Teacher Rating Scale and multiple gating procedure within elementary and middle school samples.

The primary purposes of this investigation were to (a) continue a line of research examining the psychometric defensibility of the Social, Academic, and Emotional Behavior Risk Screener - Teacher Rating Scale (SAEBRS-TRS), and (b) develop and preliminarily evaluate the diagnostic accuracy of a novel multiple gating procedure based on teacher nomination and the SAEBRS-TRS. Two studies were conducted with elementary and middle school student samples across two separate geographic locations. Study 1 (n=864 students) results supported SAEBRS-TRS defensibility, revealing acceptable to optimal levels of internal consistency reliability, concurrent validity, and diagnostic accuracy. Findings were promising for a combined multiple gating procedure, which demonstrated acceptable levels of sensitivity and specificity. Study 2 (n=1534 students), which replicated Study 1 procedures, further supported the SAEBRS-TRS' psychometric defensibility in terms of reliability, validity, and diagnostic accuracy. Despite the incorporation of revisions intended to promote sensitivity levels, the combined multiple gating procedure's diagnostic accuracy was similar to that found in Study 1. Taken together, results build upon prior research in support of the applied use of the SAEBRS-TRS, as well as justify future research regarding a SAEBRS-based multiple gating procedure. Implications for practice and study limitations are discussed.

Evaluating the applied use of a mental health screener: Structural validity of the Social, Academic, and Emotional Behavior Risk Screener.

Universal screening for mental health has gained prominence in schools with the adoption of multitiered systems of support. However, there is a general lack of brief, psychometrically defensible instruments that assess emotional and behavioral risk. This study employed a multilevel, confirmatory bifactor analysis to evaluate the factor structure of a novel screening instrument-the Social, Academic, and Emotional Behavioral Risk Screener (SAEBRS; Kilgus & von der Embse, 2014)-examining the structure at the student (within) and teacher or rater (between) levels. Item response theory (IRT) analyses were then used to examine the functioning of 2 existing factors, social risk and academic risk, in addition to a newly introduced third factor, emotional risk, within a sample of 834 elementary and middle school students. Results indicated good fit of a bifactor model including the addition of the new Emotional Behavior subscale. IRT analyses suggested strong item-level discriminative properties (a > 1.0) for 17 of the 19 SAEBRS items and indicated that scale precision was greatest within the low to moderate range of each respective dimension (social, academic, and behavioral risk). Overall, the findings provide support for the use of the SAEBRS as a screener for mental health-related concerns. Implications for model interpretation and model use are discussed. (PsycINFO Database Record

Sensitivity to change and concurrent validity of direct behavior ratings for academic anxiety.

Multitiered frameworks of service delivery have traditionally underserved students with mental health needs. Whereas research has supported the assessment and intervention of social and academic behavior across tiers, evidence is limited with regard to mental health concerns including internalizing behaviors (e.g., anxiety and depression). In particular, there is a notable shortage of brief anxiety assessment tools to be used for progress monitoring purposes. Moreover, traditional omnibus rating scale approaches may fail to capture contextually dependent anxiety. The purpose of the present investigation is to examine the sensitivity to change and concurrent validity of Direct Behavior Ratings (DBR; Chafouleas, Riley-Tillman, & Christ, 2009; Chafouleas, Riley-Tillman, & Sugai, 2007) of anxiety and traditional rating scales in measuring academic anxiety directly before, during, and after a potentially anxiety provoking stimulus. Research was conducted with 115 undergraduate students in a Southeastern university. Results indicated significant relationships between DBRs and pre- and postmeasures of anxiety. Change metrics suggested an overall lack of correspondence between DBR and the criterion measure, with DBR scales detecting greater change both across the testing situation and participants. The use of DBR for anxiety is considered within a multitiered, problem-solving framework. Feasibility and limitations associated with implementation are discussed.

Confirmation of models for interpretation and use of the Social and Academic Behavior Risk Screener (SABRS).

The purpose of this investigation was to evaluate the models for interpretation and use that serve as the foundation of an interpretation/use argument for the Social and Academic Behavior Risk Screener (SABRS). The SABRS was completed by 34 teachers with regard to 488 students in a Midwestern high school during the winter portion of the academic year. Confirmatory factor analysis supported interpretation of SABRS data, suggesting the fit of a bifactor model specifying 1 broad factor (General Behavior) and 2 narrow factors (Social Behavior [SB] and Academic Behavior [AB]). The interpretive model was further supported by analyses indicative of the internal consistency and interrater reliability of scores from each factor. In addition, latent profile analyses indicated the adequate fit of the proposed 4-profile SABRS model for use. When cross-referenced with SABRS cut scores identified via previous work, results revealed students could be categorized as (a) not at-risk on both SB and AB, (b) at-risk on SB but not on AB, (c) at-risk on AB but not on SB, or (d) at-risk on both SB and AB. Taken together, results contribute to growing evidence supporting the SABRS within universal screening. Limitations, implications for practice, and future directions for research are discussed herein.