Predictive Validity of the K-CAT-SS in High-Risk Adolescents and Young Adults.

OBJECTIVE: Suicide is a leading cause of death in adolescents and young adults and has increased substantially in the past 15 years. Accurate suicide risk stratification based on rapid screening can help reverse these trends. This study aimed to assess the ability of the Kiddie Computerized Adaptive Test Suicide Scale (K-CAT-SS), a brief computerized adaptive test of suicidality, to predict suicide attempts (SAs) in high-risk youth. METHOD: A total of 652 participants (age range, 12-24 years), 78% of whom presented with suicidal ideation or behavior, were recruited within 1 month of mental health care contact. The K-CAT-SS, scaled from 0 to 100, was administered at baseline, and participants were assessed at about 1, 3, and 6 months after intake. Weekly incidence of SAs was assessed using the Adolescent Longitudinal Interval Follow-up Evaluation and Columbia-Suicide Severity Rating Scale. A secondary outcome was suicidal behavior, including aborted, interrupted, and actual SAs. RESULTS: The K-CAT-SS showed a 4.91-fold increase in SAs for every 25-point increase in the baseline score (95% CI 2.83-8.52) and a 3.51-fold increase in suicidal behaviors (95% CI 2.32-5.30). These relations persisted following adjustment for prior attempts; demographic variables including age, sex, gender identity, sexual orientation, and race/ethnicity; and other measures of psychopathology. No moderating effects were identified. At 3 months, area under the receiver operating characteristic curve was 0.83 (95% CI 0.72-0.93) for 1 or more SAs. CONCLUSION: The K-CAT-SS is an excellent tool for suicide risk stratification, particularly in higher-risk populations where other measures have shown lower predictive validity.

Adolescent, Parent, and Provider Perceptions of a Predictive Algorithm to Identify Adolescent Suicide Risk in Primary Care.

OBJECTIVE: To understand adolescent, parent, and provider perceptions of a machine learning algorithm for detecting adolescent suicide risk prior to its implementation primary care. METHODS: We conducted semi-structured, qualitative interviews with adolescents (n = 9), parents (n = 12), and providers (n = 10; mixture of behavioral health and primary care providers) across two major health systems. Interviews were audio recorded and transcribed with analyses supported by use of NVivo. A codebook was developed combining codes derived inductively from interview transcripts and deductively from implementation science frameworks for content analysis. RESULTS: Reactions to the algorithm were mixed. While many participants expressed privacy concerns, they believed the algorithm could be clinically useful for identifying adolescents at risk for suicide and facilitating follow-up. Parents' past experiences with their adolescents' suicidal thoughts and behaviors contributed to their openness to the algorithm. Results also aligned with several key Consolidated Framework for Implementation Research domains. For example, providers mentioned barriers inherent to the primary care setting such as time and resource constraints likely to impact algorithm implementation. Participants also cited a climate of mistrust of science and health care as potential barriers. CONCLUSIONS: Findings shed light on factors that warrant consideration to promote successful implementation of suicide predictive algorithms in pediatric primary care. By attending to perspectives of potential end users prior to the development and testing of the algorithm, we can ensure that the risk prediction methods will be well-suited to the providers who would be interacting with them and the families who could benefit.

Stakeholder perspectives on digital behavioral health applications targeting adolescent depression and suicidality: Policymaker, provider, and community insights.

Background: With adolescent depression and suicidality increasing, technology-based interventions may help address mental health needs. The Enhancing Treatment and Utilization for Depression and Emergent Suicidality (ETUDES) Center supports development of technology interventions to help primary care providers target these problems. To successfully develop and integrate such interventions into primary care, iterative engagement of stakeholders is necessary. This paper outlines our stakeholder engagement process, a qualitative analysis of feedback and outcomes, and how feedback was incorporated to develop Center interventions. Methods: Stakeholder panels represented key end-users of ETUDES Center interventions (adolescents, young adults, parents, and healthcare providers) and their advocates at the organizational/policy level. Meetings were held throughout intervention development and at annual retreats. Detailed meeting notes were collected and summarized by the stakeholder engagement team in real time, after which action items were generated and reviewed by the full research team. Using a content analysis approach, all stakeholder feedback summaries were coded using a prespecified codebook organized by recurring topics. Codes were organized under the Consolidated Framework for Implementation Research (CFIR). Anonymous stakeholder feedback surveys assessed relevance of topics, meeting effectiveness, and stakeholder involvement. Results: Stakeholder meetings provided feedback on topics such as representation, language, access to care, and stigma. Relevant feedback and recommendations were incorporated into subsequent iterations of the interventions and their implementation. Mean stakeholder ratings of meeting proceedings on a 0-3 Likert scale ranged from 1.70 (SD = 0.10) for participation to 2.43 (SD = 0.08) for effectiveness in addressing meeting agenda. Conclusions: The iterative engagement approach yielded practical feedback from stakeholders about ETUDES Center interventions. The team organized feedback to identify barriers and facilitators to using Center interventions and to generate action items, which were transposed onto components of an implementation strategy, supplemented by the CFIR-ERIC Implementation Strategy Matching Tool. Stakeholder feedback will direct the future development of an integrated intervention and guide further stakeholder engagement in developing technologies for adolescent mental health.Plain Language Summary: Depression and suicide in teens have been on the rise for the past several years. Primary care may be an ideal place to address these concerns because most teens have a primary care provider (PCP) who can offer a confidential place for both teens and their caregivers. Our Center develops technology-based interventions to help PCPs address teen depression and suicidality. Multiple barriers may hinder PCPs in addressing teen depression and suicidality. This paper describes how we engaged multiple types of stakeholders-or individuals with an interest in adolescent mental health-to ask for their feedback over a series of meetings, to help us improve the fit of our interventions to this population. Stakeholders included providers, community members like parents and mental health advocates, youth, and policymakers. We learned what was most important to them-including concerns about equitable access (e.g., providing broadband internet to families who do not have it so they can still participate) and legal risks or failures for the technology to identify suicidality. In this paper, we describe our process for not only recruiting and engaging stakeholders but also for planning action based on their feedback. Similar processes could be used by other researchers and clinical organizations seeking to incorporate technology into mental health interventions.

Perceived Facilitators of and Barriers to Implementation of a Decision Support Tool for Adolescent Depression and Suicidality Screening: Focus Group and Interview Study.

BACKGROUND: Screening Wizard (SW) is a technology-based decision support tool aimed at guiding primary care providers (PCPs) to respond to depression and suicidality screens in adolescents. Separate screens assess adolescents' and parents' reports on mental health symptoms, treatment preferences, and potential treatment barriers. A detailed summary is provided to PCPs, also identifying adolescent-parent discrepancies. The goal of SW is to enhance decision-making to increase the utilization of evidence-based treatments. OBJECTIVE: This qualitative study aims to describe multi-stakeholder perspectives of adolescents, parents, and providers to understand the potential barriers to the implementation of SW. METHODS: We interviewed 11 parents and 11 adolescents and conducted two focus groups with 18 health care providers (PCPs, nurses, therapists, and staff) across 2 pediatric practices. Participants described previous experiences with screening for depression and were shown a mock-up of SW and asked for feedback. Interviews and focus groups were transcribed verbatim, and codebooks were inductively developed based on content. Transcripts were double coded, and disagreements were adjudicated to full agreement. Completed coding was used to produce thematic analyses of the interviews and focus groups. RESULTS: We identified five main themes across the interviews and focus groups: parents, adolescents, and pediatric PCPs agree that depression screening should occur in pediatric primary care; there is concern that accurate self-disclosure does not always occur during depression screening; SW is viewed as a tool that could facilitate depression screening and that might encourage more honesty in screening responses; parents, adolescents, and providers do not want SW to replace mental health discussions with providers; and providers want to maintain autonomy in treatment decisions. CONCLUSIONS: We identified that providers, parents, and adolescents are all concerned with current screening practices, mainly regarding inaccurate self-disclosure. They recognized value in SW as a computerized tool that may elicit more honest responses and identify adolescent-parent discrepancies. Surprisingly, providers did not want the SW report to include treatment recommendations, and all groups did not want the SW report to replace conversations with the PCP about depression. Although SW was originally developed as a treatment decision algorithm, this qualitative study has led us to remove this component, and instead, SW focuses on aspects identified as most useful by all groups. We hope that this initial qualitative work will improve the future implementation of SW.

Computerized Adaptive Tests for Rapid and Accurate Assessment of Psychopathology Dimensions in Youth.

OBJECTIVE: At least half of youths with mental disorders are unrecognized and untreated. Rapid, accurate assessment of child mental disorders could facilitate identification and referral and potentially reduce the occurrence of functional disability that stems from early-onset mental disorders. METHOD: Computerized adaptive tests (CATs) based on multidimensional item response theory were developed for depression, anxiety, mania/hypomania, attention-deficit/hyperactivity disorder, conduct disorder, oppositional defiant disorder, and suicidality, based on parent and child ratings of 1,060 items each. In phase 1, CATs were developed from 801 participants. In phase 2, predictive, discriminant, and convergent validity were tested against semi-structured research interviews for diagnoses and suicidality in 497 patients and 104 healthy controls. Overall strength of association was determined by area under the receiver operating characteristic curve (AUC). RESULTS: The child and parent independently completed the Kiddie-Computerized Adaptive Tests (K-CATs) in a median time of 7.56 and 5.03 minutes, respectively, with an average of 7 items per domain. The K-CATs accurately captured the presence of diagnoses (AUCs from 0.83 for generalized anxiety disorder to 0.92 for major depressive disorder) and suicidal ideation (AUC = 0.996). Strong correlations with extant measures were found (r ≥ 0.60). Test-retest reliability averaged r = 0.80. CONCLUSION: These K-CATs provide a new approach to child psychopathology screening and measurement. Testing can be completed by child and parent in less than 8 minutes and yields results that are highly convergent with much more time-consuming structured clinical interviews and dimensional severity assessment and measurement. Testing of the implementation of the K-CAT is now indicated.