The Feasibility of Embedding Data Collection into the Routine Service Delivery of a Multi-Component Program for High-Risk Young People.

BACKGROUND: There is little evidence about how to improve outcomes for high-risk young people, of whom Indigenous young people are disproportionately represented, due to few evaluation studies of interventions. One way to increase the evidence is to have researchers and service providers collaborate to embed evaluation into the routine delivery of services, so program delivery and evaluation occur simultaneously. This study aims to demonstrate the feasibility of integrating best-evidence measures into the routine data collection processes of a service for high-risk young people, and identify the number and nature of risk factors experienced by participants. METHODS: The youth service is a rural based NGO comprised of multiple program components: (i) engagement activities; (ii) case management; (iii) diversionary activities; (iv) personal development; and (v) learning and skills. A best-evidence assessment tool was developed by staff and researchers and embedded into the service's existing intake procedure. Assessment items were organised into demographic characteristics and four domains of risk: education and employment; health and wellbeing; substance use; and crime. Descriptive data are presented and summary risk variables were created for each domain of risk. A count of these summary variables represented the number of co-occurring risks experienced by each participant. The feasibility of this process was determined by the proportion of participants who completed the intake assessment and provided research consent. RESULTS: This study shows 85% of participants completed the assessment tool demonstrating that data on participant risk factors can feasibly be collected by embedding a best-evidence assessment tool into the routine data collection processes of a service. The most prevalent risk factors were school absence, unemployment, suicide ideation, mental distress, substance use, low levels of physical activity, low health service utilisation, and involvement in crime or with the juvenile justice system. All but one participant experienced at least two co-occurring domains of risk, and the majority of participants (58%) experienced co-occurring risk across four domains. CONCLUSIONS: This is the first study to demonstrate that best-evidence measures can feasibly be embedded into the routine data collection processes of a service for high-risk young people. This process allows services to tailor their activities to the most prevalent risks experienced by participants, and monitor these risks over time. Replication of this process in other services would improve the quality of services, facilitate more high quality evaluations of services, and contribute evidence on how to improve outcomes for high-risk young people.

The quality and effectiveness of interventions that target multiple risk factors among young people: a systematic review.

OBJECTIVE: To identify evaluations of interventions that target multiple risk factors in high-risk young people, describe their characteristics, critique their methodological quality and summarise their effectiveness. METHODS: A search of the literature published between 2009 and 2014 identified 13 evaluations of interventions that targeted multiple risk factors, compared to 95 evaluations that targeted single risk factors. The methodological adequacy of the 13 evaluation studies was analysed using the Quality Assessment Tool for Quantitative Studies and information regarding characteristics and intervention effectiveness was extracted and summarised. RESULTS: There were very few outcome evaluation studies of interventions that targeted multiple risk factors, relative to single risk factors, among high-risk young people. Of the identified studies, half were methodologically weak. Interventions delivered in community settings targeted a greater number of risk factors, while those delivered in a school or health setting reported a higher proportion of statistically significant outcomes. No economic analyses were conducted. Conclusions and Implications for Public Health: More methodologically rigorous evaluations of interventions targeting multiple risk factors among high-risk young people are required, especially for those delivered in community settings. Four key areas for improvement are: i) more precisely defining the risk factors experienced by high-risk young people; ii) achieving greater consistency across interventions; iii) standardising outcome measures; and iv) conducting economic analyses.