Frequent and unpredictable changes in COVID-19 policies and restrictions reduce the accuracy of model forecasts.

Between June and August 2020, an agent-based model was used to project rates of COVID-19 infection incidence and cases diagnosed as positive from 15 September to 31 October 2020 for 72 geographic settings. Five scenarios were modelled: a baseline scenario where no future changes were made to existing restrictions, and four scenarios representing small or moderate changes in restrictions at two intervals. Post hoc, upper and lower bounds for number of diagnosed Covid-19 cases were compared with actual data collected during the prediction window. A regression analysis with 17 covariates was performed to determine correlates of accurate projections. It was found that the actual data fell within the lower and upper bounds in 27 settings and out of bounds in 45 settings. The only statistically significant predictor of actual data within the predicted bounds was correct assumptions about future policy changes (OR 15.04; 95% CI 2.20-208.70; p = 0.016). Frequent changes in restrictions implemented by governments, which the modelling team was not always able to predict, in part explains why the majority of model projections were inaccurate compared with actual outcomes and supports revision of projections when policies are changed as well as the importance of modelling teams collaborating with policy experts.

Cost optimisation analysis of the expanded programme for immunisation: balancing equity and coverage in Pakistan.

INTRODUCTION: With limited resources, attaining maximal average health service coverage can be at odds with maximising equity which attempts to promote greater reach among underserved populations. In this study, we examined the trade-offs in immunisation coverage levels and equity for children under 5 years of age in Pakistan across various subpopulations who can be targeted with different combinations of immunisation service modalities. METHODS: We conducted a detailed costing exercise across 16 geographically and demographically diverse districts in Pakistan. These data were the basis for (a) technical efficiency benchmarking via Data Envelopment Analysis to identify potential efficiency gains by location, delivery model and cost ingredient; (b) allocative efficiency optimisation modelling to understand how resource allocations could be optimised and to devise recommended budget allocations and operational metrics. Finally, the hypothetical overall efficiency gains attainable were estimated if available resources were allocated with the optimal emphases, and if service delivery models operated at productivity levels at the benchmarked frontier of efficiency. RESULTS: Benchmarking suggests that ~44% of delivery models are running efficiently and 37% are highly inefficient. While coverage and equity are usually at odds, surprisingly, the optimisation modelling revealed that substantial improvements in equity between subpopulations does not necessarily cost very much in overall immunisation coverage: theoretically, equity can be achieved while still attaining close to maximal immunisation coverage. Overall, analyses suggest greater emphases should be placed on outreach delivery models which particularly target rural areas and slum populations. CONCLUSION: The unit cost differentials within districts are not sufficiently large for there to be a large reduction in potential Fully Immunised Children coverage if one focuses on maximising equity. However, reallocations of programme budgets can have a significant impact on equity outcomes, particularly at current low spending amounts. Therefore, it is recommended to address equity as the key objective in national immunisation programming.