Prioritisation processes for programme implementation and evaluation in public health: A scoping review.

Background: Programme evaluation is an essential and systematic activity for improving public health programmes through useful, feasible, ethical, and accurate methods. Finite budgets require prioritisation of which programmes can be funded, first, for implementation, and second, evaluation. While criteria for programme funding have been discussed in the literature, a similar discussion around criteria for which programmes are to be evaluated is limited. We reviewed the criteria and frameworks used for prioritisation in public health more broadly, and those used in the prioritisation of programmes for evaluation. We also report on stakeholder involvement in prioritisation processes, and evidence on the use and utility of the frameworks or sets of criteria identified. Our review aims to inform discussion around which criteria and domains are best suited for the prioritisation of public health programmes for evaluation. Methods: We reviewed the peer-reviewed literature through OVID MEDLINE (PubMed) on 11 March 2022. We also searched the grey literature through Google and across key websites including World Health Organization (WHO), US Centers for Disease Control and Prevention (CDC), European Centre for Disease Prevention and Control (ECDC), and the International Association of National Public Health Institutes (IANPHI) (14 March 2022). Articles were limited to those published between 2002 and March 2022, in English, French or German. Results: We extracted over 300 unique criteria from 40 studies included in the analysis. These criteria were categorised into 16 high-level conceptual domains to allow synthesis of the findings. The domains most frequently considered in the studies were "burden of disease" (33 studies), "social considerations" (30 studies) and "health impacts of the intervention" (28 studies). We only identified one paper which proposed criteria for use in the prioritisation of public health programmes for evaluation. Few prioritisation frameworks had evidence of use outside of the setting in which they were developed, and there was limited assessment of their utility. The existing evidence suggested that prioritisation frameworks can be used successfully in budget allocation, and have been reported to make prioritisation more robust, systematic, transparent, and collaborative. Conclusion: Our findings reflect the complexity of prioritisation in public health. Development of a framework for the prioritisation of programmes to be evaluated would fill an evidence gap, as would formal assessment of its utility. The process itself should be formal and transparent, with the aim of engaging a diverse group of stakeholders including patient/public representatives.

A Multidisciplinary Investigation into the Talent Development Processes at an English Football Academy: A Machine Learning Approach.

The talent development processes in youth football are both complex and multidimensional. The purpose of this two-fold study was to apply a multidisciplinary, machine learning approach to examine: (a) the developmental characteristics of under-9 to under-16 academy players (n = 98; Study 1), and (b) the characteristics of selected and deselected under-18 academy players (n = 18; Study 2). A combined total of 53 factors cumulated from eight data collection methods across two seasons were analysed. A cross-validated Lasso regression was implemented, using the glmnet package in R, to analyse the factors that contributed to: (a) player review ratings (Study 1), and (b) achieving a professional contract (Study 2). Results showed non-zero coefficients for improvement in subjective performance in 15 out of the 53 analysed features, with key findings revealing advanced percentage of predicted adult height (0.196), greater lob pass (0.160) and average dribble completion percentage (0.124), more total match-play hours (0.145), and an older relative age (BQ1 vs. BQ2: -0.133; BQ1 vs. BQ4: -0.060) were the most important features that contributed towards player review ratings. Moreover, PCDEQ Factor 3 and an ability to organise and engage in quality practice (PCDEQ Factor 4) were important contributing factors towards achieving a professional contract. Overall, it appears the key factors associated with positive developmental outcomes are not always technical and tactical in nature, where coaches often have their expertise. Indeed, the relative importance of these factors is likely to change over time, and with age, although psychological attributes appear to be key to reaching potential across the academy journey. The methodological techniques used here also serve as an impetus for researchers to adopt a machine learning approach when analysing multidimensional databases.

Economic impact of a rapid, on-demand ADAMTS-13 activity assay for the diagnosis of thrombotic thrombocytopenic purpura.

Background: Thrombotic thrombocytopenic purpura (TTP) is a rare, life-threatening thrombotic microangiopathy (TMA), characterized by ADAMTS-13 activity <10%. ADAMTS-13 activity assays are typically performed in reference laboratories with a turnaround time of several days. First-line treatment for TTP, therapeutic plasma exchange (TPE), typically starts while results are pending. The automated, on-demand HemosIL AcuStar ADAMTS-13 Activity assay provides results in under an hour, which could reduce unnecessary TPE use and associated costs. Objectives: To estimate the hospital budget impact in the United States, United Kingdom, and France of using a rapid ADAMTS-13 activity assay. Methods: We compared routine use of a rapid assay in adults with TMA with a scenario in which results take 3 days. Model structure and variables were based on published literature, plus survey and interviews of five clinicians from the three countries. Costs for the ADAMTS-13 activity assays and TPE were included. Results: Model results suggest that if an on-demand, rapid ADAMTS-13 activity assay is used, US, UK, and French hospitals could save $18 million, £1.2 million, and €1.6 million annually, respectively. This equates to $10 788, £3497, and €4700 saved per patient with TMA in the United States, United Kingdom, and France. The model is most sensitive to the exact split of diagnoses of TMA cases, as savings accrue from non-TTP diagnoses. Conclusions: In patients with TMA, use of a rapid, on-demand ADAMTS-13 activity assay such as the HemosIL AcuStar ADAMTS-13 Activity assay has the potential to be cost saving for hospitals.

Pedometers can help people get more active as part of an exercise programme.

The studyHodkinson A, Kontopantelis E, Adeniji C, et al. Accelerometer- and pedometer-based physical activity interventions among adults with cardiometabolic conditions: a systematic review and meta-analysis JAMA Network Open 2019;2:e1912895.This study was funded by the NIHR School for Primary Care Research (project number 390 ESWG Workstream 3).To read the full NIHR Signal, go to: https://discover.dc.nihr.ac.uk/content/signal-000880/pedometers-can-help-people-get-more-active-as-part-of-an-exercise-programme.

Computerised speech and language therapy can help people with aphasia find words following a stroke.

The studyPalmer R, Dimairo M, Cooper C, et al. Self-managed, computerised speech and language therapy for patients with chronic aphasia post-stroke compared with usual care or attention control (Big CACTUS): a multicentre, single-blinded, randomised controlled trial. Lancet Neurol 2019;18:821-33.This project was funded by the NIHR Health Technology Assessment Programme (project number 12/21/01) and the Tavistock Trust for Aphasia.To read the full NIHR Signal, go to: https://discover.dc.nihr.ac.uk/content/signal-000864/after-a-stroke-computerised-speech-and-language-therapy-can-help-people-find-words.

Online patient feedback is positive, but not used effectively.

The studyPowell J, Atherton H, Williams V, et al. Using online patient feedback to improve NHS services: the INQUIRE multimethod study. Health Serv Deliv Res 2019;7:38.This project was funded by the NIHR Health Services and Delivery Research programme (project number HS&DR 14/04/48).To read the full NIHR Signal, go to: https://discover.dc.nihr.ac.uk/content/signal-000861/online-patient-feedback-is-mostly-positive-but-is-not-being-used-effectively.

Group cognitive behavioural courses may reduce fatigue from rheumatoid arthritis.

The studyHewlett S, Almeida C, Ambler N, et al. Reducing arthritis fatigue impact: two-year randomised controlled trial of cognitive behavioural approaches by rheumatology teams (RAFT). Ann Rheum Dis 2019;78:465-72.Hewlett S, Almeida C, Ambler N, et al. Group cognitive behavioural programme to reduce the impact of rheumatoid arthritis fatigue: the RAFT RCT with economic and qualitative evaluations. Health Technol Assess 2019;23:57.This project was funded by the NIHR Health Technology Assessment Programme (project number 11/112/01).To read the full NIHR Signal, go to: https://discover.dc.nihr.ac.uk/content/signal-000860/group-cognitive-behavioural-courses-may-reduce-fatigue-from-rheumatoid-arthritis.

Tranexamic acid is safe to use following mild-to-moderate traumatic brain injury.

The studyThe CRASH-3 Trial Collaborators. Effects of tranexamic acid on death, disability, vascular occlusive events and other morbidities in patients with acute traumatic brain injury (CRASH-3): a randomised, placebo-controlled trial. Lancet 2019;394:1713-23.This trial was funded by NIHR Health Technology Assessment Programme (project number 14/190/01), JP Moulton Charitable Trust, Department of Health and Social Care, Department for International Development, Global Challenges Research Fund, Medical Research Council, and the Wellcome Trust (Joint Global Health Trials scheme).To read the full NIHR Signal, go to: https://discover.dc.nihr.ac.uk/content/signal-000870/tranexamic-acid-following-mild-to-moderate-traumatic-brain-injury-is-safe-and-reduces-deaths.

Losing weight following diagnosis of type 2 diabetes boosts chance of remission.

The studyDambha-Miller H, Day AJ, Strelitz J, et al. Behaviour change, weight loss and remission of Type 2 diabetes: a community-based prospective cohort study. Diabet Med 2019. doi:10.1111/dme.14122This project was funded by the NIHR Health Technology Assessment Programme (project number 08/116/300) as well as the Wellcome Trust (grant number: G061895), the Epidemiology Unit programme (MC_UU_12015/4), and the National Health Service R&D support funding.To read the full NIHR Signal, go to: https://discover.dc.nihr.ac.uk/content/signal-000841/weight-loss-after-type-2-diabetes-diagnosis-boosts-chance-of-remission.

Twenty miles per hour speed zones reduce the danger to pedestrians and cyclists.

The studyCleland LC, McComb L, Kee F, et al. Effects of 20 mph interventions on a range of public health outcomes: a meta-narrative evidence synthesis. J Transp Health 2019. doi:10.1016/j.jth.2019.100633This project was funded by the NIHR Policy Research Programme (project number 17/149/19).To read the full NIHR Signal, go to: https://discover.dc.nihr.ac.uk/content/signal-000853/twenty-mph-speed-zones-reduce-the-danger-to-pedestrians-and-cyclists.

Planned earlier delivery for late pre-eclampsia may be better for mothers.

The studyChappell LC, Brocklehurst P, Green ME, et al. Planned early delivery or expectant management for late preterm pre-eclampsia (PHOENIX): a randomised controlled trial. Lancet 2019;394:1181-90.This project was funded by the NIHR Health Technology Assessment Programme (project number 12/25/03).To read the full NIHR Signal, go to: https://discover.dc.nihr.ac.uk/content/signal-000838/mothers-benefit-from-a-planned-earlier-delivery-for-late-pre-eclampsia.

Keyhole hysterectomy is effective for women with heavy menstrual bleeding.

The studyCooper K, Breeman S, Scott NW, et al. Laparoscopic supracervical hysterectomy versus endometrial ablation for women with heavy menstrual bleeding (HEALTH): a parallel-group, open-label, randomised controlled trial. Lancet 2019;394:1425-36.The study was funded by the NIHR Health Technology Assessment Programme (project number 12/35/23).To read the full NIHR Signal, go to: https://discover.dc.nihr.ac.uk/content/signal-000837/keyhole-hysterectomy-is-effective-for-women-with-heavy-menstrual-bleeding.

Clinicians prescribe antibiotics for childhood respiratory tract infection based on assessment, rather than parental expectation.

The studyCabral C, Horwood J, Symonds J, et al. Understanding the influence of parent-clinician communication on antibiotic prescribing for children with respiratory tract infections in primary care: a qualitative observational study using a conversation analysis approach. BMC Fam Pract 2019;20:102.This project was funded by the NIHR School for Primary Care Research Programme (project number SPCR204).To read the full NIHR Signal, go to: https://discover.dc.nihr.ac.uk/content/signal-000829/gps-assessment-not-parental-expectation-drives-antibiotic-prescribing.

Telephone or internet delivered talking therapy can alleviate irritable bowel symptoms.

The studyEveritt H, Landau G, Little P. Therapist telephone-delivered CBT and web-based CBT compared with treatment as usual in refractory irritable bowel syndrome: the ACTIB three-arm RCT. Health Technol Assess 2019;23:1-154.This project was funded by the NIHR Health Technology Assessment Programme (project number 11/69/02).To read the full NIHR Signal, go to https://discover.dc.nihr.ac.uk/content/signal-000784/irritable-bowel-syndrome-helped-by-telephone-or-internet-cbt.

Antimicrobial central venous catheters do not reduce infections in pre-term babies.

The studyGilbert R, Brown M, Rainford N et al. Antimicrobial-impregnated central venous catheters for prevention of neonatal bloodstream infection (PREVAIL): an open-label, parallel-group, pragmatic, randomised controlled trial. Lancet Child Adolesc Health 2019;3:381-90.The study was funded by the NIHR Health Technology Assessment programme (project number 12/167/02).To read the full NIHR Signal, go to https://discover.dc.nihr.ac.uk/content/signal-000782/antimicrobial-central-venous-catheters-for-pre-term-babies-do-not-reduce-infections.

Routine use of progesterone does not prevent miscarriage.

The studyCoomarasamy A, Devall AJ, Cheed V, et al. A randomised trial of progesterone in women with bleeding in early pregnancy N Engl J Med 2019;380:1815-24.This project was funded by the NIHR Health Technology Assessment Programme (project number 12/167/26).To read the full NIHR Signal, go to https://discover.dc.nihr.ac.uk/content/signal-000792/miscarriage-not-prevented-by-routine-use-of-progesterone.

Levetiracetam is a useful alternative to phenytoin for epileptic seizures in children.

The studyLyttle MD, Rainford NEA, Gamble C, et al. Levetiracetam versus phenytoin for second-line treatment of paediatric convulsive status epilepticus (EcLiPSE): a multicentre, open-label, randomised trial. Lancet 2019;393:2125-34.This trial was funded by the NIHR Health Technology Assessment Programme (project number 12/127/134).To read the full NIHR Signal, go to https://discover.dc.nihr.ac.uk/content/signal-000790/levetiracetam-vs-phenytoin-in-stopping-childrens-prolonged-epileptic-seizures.

National quality improvement programmes need time and resources to have an impact.

The studyPeden CJ, Stephens T, Martin G et al. Effectiveness of a national quality improvement programme to improve survival after emergency abdominal surgery (EPOCH): a stepped-wedge cluster-randomised trial. Lancet 2019;393:2213-21.This project was funded by the NIHR Health Services and Delivery Research Programme (project number 12/5005/10).To read the full NIHR Signal, go to https://discover.dc.nihr.ac.uk/content/signal-000789/national-quality-improvement-programmes-need-time-and-resources-to-have-impact.

Cardiac rehabilitation for heart failure can improve quality of life and fitness.

The studyTaylor RS, Walker S, Ciani O, et al. Exercise-based cardiac rehabilitation for chronic heart failure: the EXTRAMATCH II individual participant data meta-analysis. Health Technol Assess 2019;23:1-98.This project was funded by the NIHR Health Technology Assessment Programme (project number 15/80/30).To read the full NIHR Signal, go to https://discover.dc.nihr.ac.uk/content/signal-000803/cardiac-rehabilitation-for-heart-failure-can-improve-quality-of-life-and-fitness.

Whole body MRI is effective for identifying metastatic disease in colorectal cancer patients.

The studyTaylor S, Mallett S, Beare S et al. Diagnostic accuracy of whole-body MRI versus standard imaging pathways for metastatic disease in newly diagnosed colorectal cancer: the prospective Streamline C trial. Lancet Gastroenterol Hepatol 2019;4:529-37.This project was funded by the NIHR Health Technology Assessment Programme (project number 10/68/01).To read the full NIHR Signal, go to https://discover.dc.nihr.ac.uk/content/signal-000797/identifying-metastatic-disease-in-colorectal-cancer-with-whole-body-mri.