Evaluating the scaling up of an effective implementation intervention (PACE) to increase the delivery of a mandatory physical activity policy in primary schools.

BACKGROUND: Physically Active Children in Education (PACE) is an effective implementation intervention for increasing the number of minutes classroom teachers schedule physical activity each week. To date, evaluations of PACE have included a smaller number of schools from only one region in New South Wales Australia. If PACE is to have population-wide benefits we must be able to deliver this support to a larger number of schools across multiple regions. This study aimed to evaluate the scale-up of PACE. METHODS: An uncontrolled before and after study, with 100 schools from three regions was conducted. Participating schools received PACE for approximately 12 months. We assessed the following outcomes: delivery of the evidence-based intervention (EBI) (i.e. minutes of physical activity scheduled by classroom teachers per week); delivery of the implementation strategies (i.e. reach, dose delivered, adherence and indicators of sustainability); and key determinants of implementation (i.e. acceptability of strategies and cost). Data were collected via project officer records, and principal and teacher surveys. Linear mixed models were used to assess EBI delivery by evaluating the difference in the mean minutes teachers scheduled physical activity per week from baseline to follow-up. Descriptive data were used to assess delivery of the implementation strategies and their perceived acceptability (i.e. PACE). A prospective, trial-based economic evaluation was used to assess cost. RESULTS: Delivery of the EBI was successful: teachers increas their average minutes of total physical activity scheduled across the school week by 26.8 min (95% CI: 21.2, 32.4, p < 0.001) after receiving PACE. Indicators for delivery of implementation strategies were high: 90% of consenting schools received all strategies and components (reach); 100% of strategies were delivered by the provider (dose); >50% of schools adhered to the majority of strategies (11 of the 14 components); and acceptability was > 50% agreement for all strategies. The incremental cost per additional minute of physical activity scheduled per week was $27 per school (Uncertainty Interval $24, $31). CONCLUSIONS: PACE can be successfully delivered across multiple regions and to a large number of schools. Given the ongoing and scalable benefits of PACE, it is important that we continue to extend and improve this program while considering ways to reduce the associated cost.

Economic evaluation of a multi-strategy intervention that improves school-based physical activity policy implementation.

BACKGROUND: Internationally, government policies mandating schools to provide students with opportunities to participate in physical activity are poorly implemented. The multi-component Physically Active Children in Education (PACE) intervention effectively assists schools to implement one such policy. We evaluated the value of investment by health service providers tasked with intervention delivery, and explored where adaptations might be targeted to reduce program costs for scale-up. METHODS: A prospective trial-based economic evaluation of an implementation intervention in 61 primary schools in New South Wales (NSW), Australia. Schools were randomised to the PACE intervention or a wait-list control. PACE strategies included centralised technical assistance, ongoing consultation, principal's mandated change, identifying and preparing in-school champions, educational outreach visits, and provision of educational materials and equipment. Effectiveness was measured as the mean weekly minutes of physical activity implemented by classroom teachers, recorded in a daily log book at baseline and 12-month follow-up. Delivery costs (reported in $AUD, 2018) were evaluated from a public finance perspective. Cost data were used to calculate: total intervention cost, cost per strategy and incremental cost (overall across all schools and as an average per school). Incremental cost-effectiveness ratios (ICERs) were calculated as the incremental cost of delivering PACE divided by the estimated intervention effect. RESULTS: PACE cost the health service provider a total of $35,692 (95% uncertainty interval [UI] $32,411, $38,331) to deliver; an average cost per school of $1151 (95%UI $1046, $1236). Training in-school champions was the largest contributor: $19,437 total; $627 ($0 to $648) average per school. Educational outreach was the second largest contributor: $4992 total; $161 ($0 to $528) average per school. The ICER was $29 (95%UI $17, $64) for every additional minute of weekly physical activity implemented per school. CONCLUSION: PACE is a potentially cost-effective intervention for increasing schools implementation of a policy mandate. The investment required by the health service provider makes use of existing funding and infrastructure; the additional cost to assist schools to implement the policy is likely not that much. PACE strategies may be adapted to substantially improve delivery costs. TRIAL REGISTRATION: Australia New Zealand Clinical Trials Registry ACTRN12617001265369; Prospectively registered 1st September 2017 https://www.anzctr.org.au/Trial/Registration/TrialReview.aspx?id=373520.

Adaptation and Validation of the Program Sustainability Assessment Tool (PSAT) for Use in the Elementary School Setting.

There is a lack of valid and reliable measures of determinants of sustainability specific to public health interventions in the elementary school setting. This study aimed to adapt and evaluate the Program Sustainability Assessment Tool (PSAT) for use in this setting. An expert reference group adapted the PSAT to ensure face validity. Elementary school teachers participating in a multi-component implementation intervention to increase their scheduling of physical activity completed the adapted PSAT. Structural validity was assessed via confirmatory factor analysis. Convergent validity was assessed using linear mixed regression evaluating the associations between scheduling of physical activity and adapted PSAT scores. Cronbach's alpha was used to evaluate internal consistency and intracluster correlation coefficients for interrater reliability. Floor and ceiling effects were also evaluated. Following adaptation and psychometric evaluation, the final measure contained 26 items. Domain Cronbach's alpha ranged from 0.77 to 0.92. Only one domain illustrated acceptable interrater reliability. Evidence for structural validity was mixed and was lacking for convergent validity. There were no floor and ceiling effects. Efforts to adapt and validate the PSAT for the elementary school setting were mixed. Future work to develop and improve measures specific to public health program sustainment that are relevant and psychometrically robust for elementary school settings are needed.