Altering the availability or proximity of food, alcohol, and tobacco products to change their selection and consumption.

BACKGROUND: Overconsumption of food, alcohol, and tobacco products increases the risk of non-communicable diseases. Interventions to change characteristics of physical micro-environments where people may select or consume these products - including shops, restaurants, workplaces, and schools - are of considerable public health policy and research interest. This review addresses two types of intervention within such environments: altering the availability (the range and/or amount of options) of these products, or their proximity (the distance at which they are positioned) to potential consumers. OBJECTIVES: 1. To assess the impact on selection and consumption of altering the availability or proximity of (a) food (including non-alcoholic beverages), (b) alcohol, and (c) tobacco products.2. To assess the extent to which the impact of these interventions is modified by characteristics of: i. studies, ii. interventions, and iii. SEARCH METHODS: We searched CENTRAL, MEDLINE, Embase, PsycINFO, and seven other published or grey literature databases, as well as trial registries and key websites, up to 23 July 2018, followed by citation searches. SELECTION CRITERIA: We included randomised controlled trials with between-participants (parallel group) or within-participants (cross-over) designs. Eligible studies compared effects of exposure to at least two different levels of availability of a product or its proximity, and included a measure of selection or consumption of the manipulated product. DATA COLLECTION AND ANALYSIS: We used a novel semi-automated screening workflow and applied standard Cochrane methods to select eligible studies, collect data, and assess risk of bias. In separate analyses for availability interventions and proximity interventions, we combined results using random-effects meta-analysis and meta-regression models to estimate summary effect sizes (as standardised mean differences (SMDs)) and to investigate associations between summary effect sizes and selected study, intervention, or participant characteristics. We rated the certainty of evidence for each outcome using GRADE. MAIN RESULTS: We included 24 studies, with the majority (20/24) giving concerns about risk of bias. All of the included studies investigated food products; none investigated alcohol or tobacco. The majority were conducted in laboratory settings (14/24), with adult participants (17/24), and used between-participants designs (19/24). All studies were conducted in high-income countries, predominantly in the USA (14/24).Six studies investigated availability interventions, of which two changed the absolute number of different options available, and four altered the relative proportion of less-healthy (to healthier) options. Most studies (4/6) manipulated snack foods or drinks. For selection outcomes, meta-analysis of three comparisons from three studies (n = 154) found that exposure to fewer options resulted in a large reduction in selection of the targeted food(s): SMD -1.13 (95% confidence interval (CI) -1.90 to -0.37) (low certainty evidence). For consumption outcomes, meta-analysis of three comparisons from two studies (n = 150) found that exposure to fewer options resulted in a moderate reduction in consumption of those foods, but with considerable uncertainty: SMD -0.55 (95% CI -1.27 to 0.18) (low certainty evidence).Eighteen studies investigated proximity interventions. Most (14/18) changed the distance at which a snack food or drink was placed from the participants, whilst four studies changed the order of meal components encountered along a line. For selection outcomes, only one study with one comparison (n = 41) was identified, which found that food placed farther away resulted in a moderate reduction in its selection: SMD -0.65 (95% CI -1.29 to -0.01) (very low certainty evidence). For consumption outcomes, meta-analysis of 15 comparisons from 12 studies (n = 1098) found that exposure to food placed farther away resulted in a moderate reduction in its consumption: SMD -0.60 (95% CI -0.84 to -0.36) (low certainty evidence). Meta-regression analyses indicated that this effect was greater: the farther away the product was placed; when only the targeted product(s) was available; when participants were of low deprivation status; and when the study was at high risk of bias. AUTHORS' CONCLUSIONS: The current evidence suggests that changing the number of available food options or altering the positioning of foods could contribute to meaningful changes in behaviour, justifying policy actions to promote such changes within food environments. However, the certainty of this evidence as assessed by GRADE is low or very low. To enable more certain and generalisable conclusions about these potentially important effects, further research is warranted in real-world settings, intervening across a wider range of foods - as well as alcohol and tobacco products - and over sustained time periods.

Altering the availability or proximity of food, alcohol, and tobacco products to change their selection and consumption.

BACKGROUND: Overconsumption of food, alcohol, and tobacco products increases the risk of non-communicable diseases. Interventions to change characteristics of physical micro-environments where people may select or consume these products - including shops, restaurants, workplaces, and schools - are of considerable public health policy and research interest. This review addresses two types of intervention within such environments: altering the availability (the range and/or amount of options) of these products, or their proximity (the distance at which they are positioned) to potential consumers. OBJECTIVES: 1. To assess the impact on selection and consumption of altering the availability or proximity of (a) food (including non-alcoholic beverages), (b) alcohol, and (c) tobacco products.2. To assess the extent to which the impact of these interventions is modified by characteristics of: i. studies, ii. interventions, and iii. SEARCH METHODS: We searched CENTRAL, MEDLINE, Embase, PsycINFO, and seven other published or grey literature databases, as well as trial registries and key websites, up to 23 July 2018, followed by citation searches. SELECTION CRITERIA: We included randomised controlled trials with between-participants (parallel group) or within-participants (cross-over) designs. Eligible studies compared effects of exposure to at least two different levels of availability of a product or its proximity, and included a measure of selection or consumption of the manipulated product. DATA COLLECTION AND ANALYSIS: We used a novel semi-automated screening workflow and applied standard Cochrane methods to select eligible studies, collect data, and assess risk of bias. In separate analyses for availability interventions and proximity interventions, we combined results using random-effects meta-analysis and meta-regression models to estimate summary effect sizes (as standardised mean differences (SMDs)) and to investigate associations between summary effect sizes and selected study, intervention, or participant characteristics. We rated the certainty of evidence for each outcome using GRADE. MAIN RESULTS: We included 24 studies, with the majority (20/24) giving concerns about risk of bias. All of the included studies investigated food products; none investigated alcohol or tobacco. The majority were conducted in laboratory settings (14/24), with adult participants (17/24), and used between-participants designs (19/24). All studies were conducted in high-income countries, predominantly in the USA (14/24).Six studies investigated availability interventions, of which two changed the absolute number of different options available, and four altered the relative proportion of less-healthy (to healthier) options. Most studies (4/6) manipulated snack foods or drinks. For selection outcomes, meta-analysis of three comparisons from three studies (n = 154) found that exposure to fewer options resulted in a large reduction in selection of the targeted food(s): SMD -1.13 (95% confidence interval (CI) -1.90 to -0.37) (low certainty evidence). For consumption outcomes, meta-analysis of three comparisons from two studies (n = 150) found that exposure to fewer options resulted in a moderate reduction in consumption of those foods, but with considerable uncertainty: SMD -0.55 (95% CI -1.27 to 0.18) (low certainty evidence).Eighteen studies investigated proximity interventions. Most (14/18) changed the distance at which a snack food or drink was placed from the participants, whilst four studies changed the order of meal components encountered along a line. For selection outcomes, only one study with one comparison (n = 41) was identified, which found that food placed farther away resulted in a moderate reduction in its selection: SMD -0.65 (95% CI -1.29 to -0.01) (very low certainty evidence). For consumption outcomes, meta-analysis of 15 comparisons from 12 studies (n = 1098) found that exposure to food placed farther away resulted in a moderate reduction in its consumption: SMD -0.60 (95% CI -0.84 to -0.36) (low certainty evidence). Meta-regression analyses indicated that this effect was greater: the farther away the product was placed; when only the targeted product(s) was available; when participants were of low deprivation status; and when the study was at high risk of bias. AUTHORS' CONCLUSIONS: The current evidence suggests that changing the number of available food options or altering the positioning of foods could contribute to meaningful changes in behaviour, justifying policy actions to promote such changes within food environments. However, the certainty of this evidence as assessed by GRADE is low or very low. To enable more certain and generalisable conclusions about these potentially important effects, further research is warranted in real-world settings, intervening across a wider range of foods - as well as alcohol and tobacco products - and over sustained time periods.

The crime, mental health, and economic impacts of prearrest diversion of people with mental health problems: A systematic review.

BACKGROUND: Prearrest diversion strategies are being adopted across the Western world, enabling the police to identify and divert people suspected of having mental disorder towards health and community services rather than the criminal justice system. AIMS: To quantify longer-term criminal justice and mental health outcomes after prearrest diversion of people with suspected mental disorder and consider economic correlates. METHODS: A systematic review of published literature on longer term outcomes after prearrest diversion. RESULTS: Only two quasi-experimental studies, with four independent samples, could be included. Findings for criminal and mental health outcomes were inconclusive, but potential for adverse outcomes was identified. Ten studies with cost data suggested that prearrest diversion can lead to overall cost savings. CONCLUSIONS: There is still inadequate evidence on which to base prearrest diversion programmes. Although some benefits have been identified by the review, so have possible harms. Future research and funding strategies must build in high-quality, systematic evaluation of outcomes before implementing a theoretically attractive strategy more widely.

What do we know about the effects of exposure to 'Low alcohol' and equivalent product labelling on the amounts of alcohol, food and tobacco people select and consume? A systematic review.

BACKGROUND: Explicit labelling of lower strength alcohol products could reduce alcohol consumption by attracting more people to buy and drink such products instead of higher strength ones. Alternatively, it may lead to more consumption due to a 'self-licensing' mechanism. Equivalent labelling of food or tobacco (for example "Low fat" or "Low tar") could influence consumption of those products by similar mechanisms. This systematic review examined the effects of 'Low alcohol' and equivalent labelling of alcohol, food and tobacco products on selection, consumption, and perceptions of products among adults. METHODS: A systematic review was conducted based on Cochrane methods. Electronic and snowball searches identified 26 eligible studies. Evidence from 12 randomised controlled trials (all on food) was assessed for risk of bias, synthesised using random effects meta-analysis, and interpreted in conjunction with evidence from 14 non-randomised studies (one on alcohol, seven on food and six on tobacco). Outcomes assessed were: quantities of the product (i) selected or (ii) consumed (primary outcomes - behaviours), (iii) intentions to select or consume the product, (iv) beliefs associated with it consumption, (v) product appeal, and (vi) understanding of the label (secondary outcomes - cognitions). RESULTS: Evidence for impacts on the primary outcomes (i.e. amounts selected or consumed) was overall of very low quality, showing mixed effects, likely to vary by specific label descriptors, products and population characteristics. Overall very low quality evidence suggested that exposure to 'Low alcohol' and equivalent labelling on alcohol, food and tobacco products can shift consumer perceptions of products, with the potential to 'self-licence' excess consumption. CONCLUSIONS: Considerable uncertainty remains about the effects of labels denoting low alcohol, and equivalent labels, on alcohol, food and tobacco selection and consumption. Independent, high-quality studies are urgently needed to inform policies on labelling regulations.

Immediate effects of alcohol marketing communications and media portrayals on consumption and cognition: a systematic review and meta-analysis of experimental studies.

BACKGROUND: Restricting marketing of alcoholic products is purported to be a cost-effective intervention to reduce alcohol consumption. The strength of evidence supporting this claim is contested. This systematic review aimed to assess immediate effects of exposure to alcohol marketing on alcoholic beverage consumption and related cognitions. METHODS: Electronic searches of nine databases, supplemented with reference list searches and forward citation tracking, were used to identify randomised, experimental studies assessing immediate effects of exposure to alcohol marketing communications on objective alcohol consumption (primary outcome), explicit or implicit alcohol-related cognitions, or selection without purchasing (secondary outcomes). Study limitations were assessed using the Cochrane Risk of Bias tool. Random and fixed effects meta-analyses were conducted to estimate effect sizes. RESULTS: Twenty four studies met the eligibility criteria. A meta-analysis integrating seven studies (758 participants, all students) found that viewing alcohol advertisements increased immediate alcohol consumption relative to viewing non-alcohol advertisements (SMD = 0.20, 95 % CI = 0.05, 0.34). A meta-analysis integrating six studies (631 participants, all students) did not find that viewing alcohol portrayals in television programmes or films increased consumption (SMD = 0.16, 95 % CI = -0.05, 0.37). Meta-analyses of secondary outcome data found that exposure to alcohol portrayals increased explicit alcohol-related cognitions, but did not find that exposure to alcohol advertisements influenced explicit or implicit alcohol-related cognitions. Confidence in results is diminished by underpowered analyses and unclear risk of bias. CONCLUSIONS: Viewing alcohol advertisements (but not alcohol portrayals) may increase immediate alcohol consumption by small amounts, equivalent to between 0.39 and 2.67 alcohol units for males and between 0.25 and 1.69 units for females. The generalizability of this finding beyond students and to other marketing channels remains to be established.

Personal financial incentives for changing habitual health-related behaviors: A systematic review and meta-analysis.

OBJECTIVES: Uncertainty remains about whether personal financial incentives could achieve sustained changes in health-related behaviors that would reduce the fast-growing global non-communicable disease burden. This review aims to estimate whether: i. financial incentives achieve sustained changes in smoking, eating, alcohol consumption and physical activity; ii. effectiveness is modified by (a) the target behavior, (b) incentive value and attainment certainty, (c) recipients' deprivation level. METHODS: Multiple sources were searched for trials offering adults financial incentives and assessing outcomes relating to pre-specified behaviors at a minimum of six months from baseline. Analyses included random-effects meta-analyses and meta-regressions grouped by timed endpoints. RESULTS: Of 24,265 unique identified articles, 34 were included in the analysis. Financial incentives increased behavior-change, with effects sustained until 18months from baseline (OR: 1.53, 95% CI 1.05-2.23) and three months post-incentive removal (OR: 2.11, 95% CI 1.21-3.67). High deprivation increased incentive effects (OR: 2.17; 95% CI 1.22-3.85), but only at >6-12months from baseline. Other assessed variables did not independently modify effects at any time-point. CONCLUSIONS: Personal financial incentives can change habitual health-related behaviors and help reduce health inequalities. However, their role in reducing disease burden is potentially limited given current evidence that effects dissipate beyond three months post-incentive removal.

Portion, package or tableware size for changing selection and consumption of food, alcohol and tobacco.

BACKGROUND: Overeating and harmful alcohol and tobacco use have been linked to the aetiology of various non-communicable diseases, which are among the leading global causes of morbidity and premature mortality. As people are repeatedly exposed to varying sizes and shapes of food, alcohol and tobacco products in environments such as shops, restaurants, bars and homes, this has stimulated public health policy interest in product size and shape as potential targets for intervention. OBJECTIVES: 1) To assess the effects of interventions involving exposure to different sizes or sets of physical dimensions of a portion, package, individual unit or item of tableware on unregulated selection or consumption of food, alcohol or tobacco products in adults and children.2) To assess the extent to which these effects may be modified by study, intervention and participant characteristics. SEARCH METHODS: We searched CENTRAL, MEDLINE, EMBASE, PsycINFO, eight other published or grey literature databases, trial registries and key websites up to November 2012, followed by citation searches and contacts with study authors. This original search identified eligible studies published up to July 2013, which are fully incorporated into the review. We conducted an updated search up to 30 January 2015 but further eligible studies are not yet fully incorporated due to their minimal potential to change the conclusions. SELECTION CRITERIA: Randomised controlled trials with between-subjects (parallel-group) or within-subjects (cross-over) designs, conducted in laboratory or field settings, in adults or children. Eligible studies compared at least two groups of participants, each exposed to a different size or shape of a portion of a food (including non-alcoholic beverages), alcohol or tobacco product, its package or individual unit size, or of an item of tableware used to consume it, and included a measure of unregulated selection or consumption of food, alcohol or tobacco. DATA COLLECTION AND ANALYSIS: We applied standard Cochrane methods to select eligible studies for inclusion and to collect data and assess risk of bias. We calculated study-level effect sizes as standardised mean differences (SMDs) between comparison groups, measured as quantities selected or consumed. We combined these results using random-effects meta-analysis models to estimate summary effect sizes (SMDs with 95% confidence intervals (CIs)) for each outcome for size and shape comparisons. We rated the overall quality of evidence using the GRADE system. Finally, we used meta-regression analysis to investigate statistical associations between summary effect sizes and variant study, intervention or participant characteristics. MAIN RESULTS: The current version of this review includes 72 studies, published between 1978 and July 2013, assessed as being at overall unclear or high risk of bias with respect to selection and consumption outcomes. Ninety-six per cent of included studies (69/72) manipulated food products and 4% (3/72) manipulated cigarettes. No included studies manipulated alcohol products. Forty-nine per cent (35/72) manipulated portion size, 14% (10/72) package size and 21% (15/72) tableware size or shape. More studies investigated effects among adults (76% (55/72)) than children and all studies were conducted in high-income countries - predominantly in the USA (81% (58/72)). Sources of funding were reported for the majority of studies, with no evidence of funding by agencies with possible commercial interests in their results.A meta-analysis of 86 independent comparisons from 58 studies (6603 participants) found a small to moderate effect of portion, package, individual unit or tableware size on consumption of food (SMD 0.38, 95% CI 0.29 to 0.46), providing moderate quality evidence that exposure to larger sizes increased quantities of food consumed among children (SMD 0.21, 95% CI 0.10 to 0.31) and adults (SMD 0.46, 95% CI 0.40 to 0.52). The size of this effect suggests that, if sustained reductions in exposure to larger-sized food portions, packages and tableware could be achieved across the whole diet, this could reduce average daily energy consumed from food by between 144 and 228 kcal (8.5% to 13.5% from a baseline of 1689 kcal) among UK children and adults. A meta-analysis of six independent comparisons from three studies (108 participants) found low quality evidence for no difference in the effect of cigarette length on consumption (SMD 0.25, 95% CI -0.14 to 0.65).One included study (50 participants) estimated a large effect on consumption of exposure to differently shaped tableware (SMD 1.17, 95% CI 0.57 to 1.78), rated as very low quality evidence that exposure to shorter, wider bottles (versus taller, narrower bottles) increased quantities of water consumed by young adult participants.A meta-analysis of 13 independent comparisons from 10 studies (1164 participants) found a small to moderate effect of portion or tableware size on selection of food (SMD 0.42, 95% CI 0.24 to 0.59), rated as moderate quality evidence that exposure to larger sizes increased the quantities of food people selected for subsequent consumption. This effect was present among adults (SMD 0.55, 95% CI 0.35 to 0.75) but not children (SMD 0.14, 95% CI -0.06 to 0.34).In addition, a meta-analysis of three independent comparisons from three studies (232 participants) found a very large effect of exposure to differently shaped tableware on selection of non-alcoholic beverages (SMD 1.47, 95% CI 0.52 to 2.43), rated as low quality evidence that exposure to shorter, wider (versus taller, narrower) glasses or bottles increased the quantities selected for subsequent consumption among adults (SMD 2.31, 95% CI 1.79 to 2.83) and children (SMD 1.03, 95% CI 0.41 to 1.65). AUTHORS' CONCLUSIONS: This review found that people consistently consume more food and drink when offered larger-sized portions, packages or tableware than when offered smaller-sized versions. This suggests that policies and practices that successfully reduce the size, availability and appeal of larger-sized portions, packages, individual units and tableware can contribute to meaningful reductions in the quantities of food (including non-alcoholic beverages) people select and consume in the immediate and short term. However, it is uncertain whether reducing portions at the smaller end of the size range can be as effective in reducing food consumption as reductions at the larger end of the range. We are unable to highlight clear implications for tobacco or alcohol policy due to identified gaps in the current evidence base.

Use and cumulation of evidence from modelling studies to inform policy on food taxes and subsidies: biting off more than we can chew?

BACKGROUND: Food tax-subsidy policies are proposed to hold promise for helping to produce healthier patterns of food purchasing and consumption at population level. Evidence for their effects derives largely from simulation studies that explore the potential effects of untried policies using a mathematical modelling framework. This paper provides a critique first of the nature of the evidence derived from such simulation studies, and second of the challenges of cumulating that evidence to inform public health policy. DISCUSSION: Effects estimated by simulation studies of food taxes and subsidies can be expected to diverge in potentially important ways from those that would accrue in practice because these models are simplified, typically static, representations of complex adaptive systems. The level of confidence that can be placed in modelled estimates of effects is correspondingly low, and the level of associated uncertainty is high. Moreover, evidence from food tax-subsidy simulation studies cannot meaningfully be cumulated using currently available quantitative evidence synthesis methods, to reduce uncertainty about effects. Simulation studies are critical for the initial phases of an incremental research process, for drawing together diverse evidence and exploring potential longer-term effects. While simulation studies of food taxes and subsidies provide a valuable and necessary input to the formulation of public health policy in this area, they are unlikely to be sufficient, and policy makers should not place excessive reliance on evidence from such studies, either singly or cumulatively. To reflect known and unknown limitations of the models, results of such studies should be interpreted cautiously as tentative projections. Modelling studies should increasingly be integrated with more empirical studies of the effects of food tax and subsidy policies in practice.

Methods used to conceptualize dimensions of health equity impacts of public health interventions in systematic reviews.

OBJECTIVES: Our aims were to, first, identify and summarize the use of methods, frameworks, and tools as a conceptual basis for investigating dimensions of equity impacts of public health interventions in systematic reviews including an equity focus. These include PROGRESS-Plus, which identifies key sociodemographic characteristics that determine health outcomes. Second, we aimed to document challenges and opportunities encountered in the application of such methods, as reported in systematic reviews. STUDY DESIGN AND SETTING: We conducted a methodological study, comprising an overview of systematic reviews with a focus on, or that aimed to assess, the equity impacts of public health interventions. We used electronic searches of the Cochrane Database of Systematic Reviews, the Database of Promoting Health Effectiveness Reviews (DoPHER), and the Finding Accessible Inequalities Research in Public Health Database, supplemented with automated searches of the OpenAlex dataset. An active learning algorithm was used to prioritize title-abstract records for manual screening against eligibility criteria. We extracted and analyzed a core dataset from a purposively selected sample of reviews, to summarize key characteristics and approaches to conceptualizing investigations of equity. RESULTS: We assessed 322 full-text reports for eligibility, from which we included 120 reports of systematic reviews. PROGRESS-Plus was the only formalized framework used to conceptualize dimensions of equity impacts. Most reviews were able to apply their intended methods to at least some degree. Where intended methods were unable to be applied fully, this was usually because primary research studies did not report the necessary information. A general rationale for focusing on equity impacts was often included, but few reviews explicitly justified their focus on (or exclusion of) specific dimensions. In addition to practical challenges such as data not being available, authors highlighted significant measurement and conceptual issues with applying these methods which may impair the ability to investigate and interpret differential impacts within and between studies. These issues included investigating constructs that lack standardized operationalization and measurement, and the complex nature of differential impacts, with dimensions that may interact with one another, as well as with particular temporal, personal, social or geographic contexts. CONCLUSION: PROGRESS-Plus is the predominant framework used in systematic reviews to conceptualize differential impacts of public health interventions by dimensions of equity. It appears sufficiently broad to encompass dimensions of equity examined in most investigations of this kind. However, PROGRESS-Plus does not necessarily ensure or guide critical thinking about more complex pathways, including interactions between dimensions of equity, and with wider contextual factors, and important practical, measurement and conceptual challenges remain. The findings from investigations of equity impacts in systematic reviews could be made more useful through more explicitly rationalized and considered approaches to the design, conduct and reporting of both primary research and the reviews themselves.

Altering micro-environments to change population health behaviour: towards an evidence base for choice architecture interventions.

BACKGROUND: The idea that behaviour can be influenced at population level by altering the environments within which people make choices (choice architecture) has gained traction in policy circles. However, empirical evidence to support this idea is limited, especially its application to changing health behaviour. We propose an evidence-based definition and typology of choice architecture interventions that have been implemented within small-scale micro-environments and evaluated for their effects on four key sets of health behaviours: diet, physical activity, alcohol and tobacco use. DISCUSSION: We argue that the limitations of the evidence base are due not simply to an absence of evidence, but also to a prior lack of definitional and conceptual clarity concerning applications of choice architecture to public health intervention. This has hampered the potential for systematic assessment of existing evidence. By seeking to address this issue, we demonstrate how our definition and typology have enabled systematic identification and preliminary mapping of a large body of available evidence for the effects of choice architecture interventions. We discuss key implications for further primary research, evidence synthesis and conceptual development to support the design and evaluation of such interventions. SUMMARY: This conceptual groundwork provides a foundation for future research to investigate the effectiveness of choice architecture interventions within micro-environments for changing health behaviour. The approach we used may also serve as a template for mapping other under-explored fields of enquiry.

GRADE guidance 23: considering cost-effectiveness evidence in moving from evidence to health-related recommendations.

BACKGROUND: This is the 23rd in a series of articles describing the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach to grading the certainty of evidence and strength of recommendations for systematic reviews, health technology assessments, and clinical guideline development. OBJECTIVES: We outline how resource utilization and cost-effectiveness analyses are integrated into health-related recommendations, using the GRADE Evidence to Decision (EtD) frameworks. STUDY DESIGN AND SETTING: Through iterative discussions and refinement, in-person, and online meetings, and through e-mail communication, we developed draft guidance to incorporate economic evidence in the formulation of health-related recommendations. We developed scenarios to operationalize the guidance. We presented a summary of the results to members of the GRADE Economic Evaluation Project Group. RESULTS: We describe how to estimate the cost of preventing (or achieving) an event to inform assessments of cost-effectiveness of alternative treatments, when there are no published economic evaluations. Evidence profiles and Summary of Findings tables based on systematic reviews of cost-effectiveness analyses can be created to provide top-level summaries of results and quality of multiple published economic evaluations. We also describe how this information could be integrated in GRADE's EtD frameworks to inform health-related recommendations. Three scenarios representing various levels of available cost-effectiveness evidence were used to illustrate the integration process. CONCLUSION: This GRADE guidance provides practical information for presenting cost-effectiveness data and its integration in the development of health-related recommendations, using the EtD frameworks.

Methods used to identify and measure resource use in economic evaluations: a systematic review of questionnaires for older people.

This paper presents the findings of a systematic review of full or partial economic evaluations that included questions to service users or their carers to elicit information on the types, amounts or costs of community-based formal social care support provided to people 65 years and older. We have found that studies seldom report use of published validated questions for eliciting information from older people in the UK about their use of formal social care services. Given the political prominence of the debate over funding social care for older people, there remains a need for analysis of policy options. This requires reliable data on the receipt and payment for care. We recommend the development of improved questions on care that are clear, robust and up-to-date with developments in policy and practice.

Machine learning reduced workload for the Cochrane COVID-19 Study Register: development and evaluation of the Cochrane COVID-19 Study Classifier.

BACKGROUND: This study developed, calibrated and evaluated a machine learning (ML) classifier designed to reduce study identification workload in maintaining the Cochrane COVID-19 Study Register (CCSR), a continuously updated register of COVID-19 research studies. METHODS: A ML classifier for retrieving COVID-19 research studies (the 'Cochrane COVID-19 Study Classifier') was developed using a data set of title-abstract records 'included' in, or 'excluded' from, the CCSR up to 18th October 2020, manually labelled by information and data curation specialists or the Cochrane Crowd. The classifier was then calibrated using a second data set of similar records 'included' in, or 'excluded' from, the CCSR between October 19 and December 2, 2020, aiming for 99% recall. Finally, the calibrated classifier was evaluated using a third data set of similar records 'included' in, or 'excluded' from, the CCSR between the 4th and 19th of January 2021. RESULTS: The Cochrane COVID-19 Study Classifier was trained using 59,513 records (20,878 of which were 'included' in the CCSR). A classification threshold was set using 16,123 calibration records (6005 of which were 'included' in the CCSR) and the classifier had a precision of 0.52 in this data set at the target threshold recall >0.99. The final, calibrated COVID-19 classifier correctly retrieved 2285 (98.9%) of 2310 eligible records but missed 25 (1%), with a precision of 0.638 and a net screening workload reduction of 24.1% (1113 records correctly excluded). CONCLUSIONS: The Cochrane COVID-19 Study Classifier reduces manual screening workload for identifying COVID-19 research studies, with a very low and acceptable risk of missing eligible studies. It is now deployed in the live study identification workflow for the Cochrane COVID-19 Study Register.

Machine learning reduced workload with minimal risk of missing studies: development and evaluation of a randomized controlled trial classifier for Cochrane Reviews.

OBJECTIVES: This study developed, calibrated, and evaluated a machine learning classifier designed to reduce study identification workload in Cochrane for producing systematic reviews. METHODS: A machine learning classifier for retrieving randomized controlled trials (RCTs) was developed (the "Cochrane RCT Classifier"), with the algorithm trained using a data set of title-abstract records from Embase, manually labeled by the Cochrane Crowd. The classifier was then calibrated using a further data set of similar records manually labeled by the Clinical Hedges team, aiming for 99% recall. Finally, the recall of the calibrated classifier was evaluated using records of RCTs included in Cochrane Reviews that had abstracts of sufficient length to allow machine classification. RESULTS: The Cochrane RCT Classifier was trained using 280,620 records (20,454 of which reported RCTs). A classification threshold was set using 49,025 calibration records (1,587 of which reported RCTs), and our bootstrap validation found the classifier had recall of 0.99 (95% confidence interval 0.98-0.99) and precision of 0.08 (95% confidence interval 0.06-0.12) in this data set. The final, calibrated RCT classifier correctly retrieved 43,783 (99.5%) of 44,007 RCTs included in Cochrane Reviews but missed 224 (0.5%). Older records were more likely to be missed than those more recently published. CONCLUSIONS: The Cochrane RCT Classifier can reduce manual study identification workload for Cochrane Reviews, with a very low and acceptable risk of missing eligible RCTs. This classifier now forms part of the Evidence Pipeline, an integrated workflow deployed within Cochrane to help improve the efficiency of the study identification processes that support systematic review production.

An evaluation of Cochrane Crowd found that crowdsourcing produced accurate results in identifying randomized trials.

BACKGROUND AND OBJECTIVES: Filtering the deluge of new research to facilitate evidence synthesis has proven to be unmanageable using current paradigms of search and retrieval. Crowdsourcing, a way of harnessing the collective effort of a "crowd" of people, has the potential to support evidence synthesis by addressing this information overload created by the exponential growth in primary research outputs. Cochrane Crowd, Cochrane's citizen science platform, offers a range of tasks aimed at identifying studies related to health care. Accompanying each task are brief, interactive training modules, and agreement algorithms that help ensure accurate collective decision-making.The aims of the study were to evaluate the performance of Cochrane Crowd in terms of its accuracy, capacity, and autonomy and to examine contributor engagement across three tasks aimed at identifying randomized trials. STUDY DESIGN AND SETTING: Crowd accuracy was evaluated by measuring the sensitivity and specificity of crowd screening decisions on a sample of titles and abstracts, compared with "quasi gold-standard" decisions about the same records using the conventional methods of dual screening. Crowd capacity, in the form of output volume, was evaluated by measuring the number of records processed by the crowd, compared with baseline. Crowd autonomy, the capability of the crowd to produce accurate collectively derived decisions without the need for expert resolution, was measured by the proportion of records that needed resolving by an expert. RESULTS: The Cochrane Crowd community currently has 18,897 contributors from 163 countries. Collectively, the Crowd has processed 1,021,227 records, helping to identify 178,437 reports of randomized controlled trials (RCTs) for Cochrane's Central Register of Controlled Trials. The sensitivity for each task was 99.1% for the RCT identification task (RCT ID), 99.7% for the RCT identification task of trials from ClinicalTrials.gov (CT ID), and 97.7% for the identification of RCTs from the International Clinical Trials Registry Platform (ICTRP ID). The specificity for each task was 99% for RCT ID, 98.6% for CT ID, and 99.1% for CT ICTRP ID. The capacity of the combined Crowd and machine learning workflow has increased fivefold in 6 years, compared with baseline. The proportion of records requiring expert resolution across the tasks ranged from 16.6% to 19.7%. CONCLUSION: Cochrane Crowd is sufficiently accurate and scalable to keep pace with the current rate of publication (and registration) of new primary studies. It has also proved to be a popular, efficient, and accurate way for a large number of people to play an important voluntary role in health evidence production. Cochrane Crowd is now an established part of Cochrane's effort to manage the deluge of primary research being produced.

Cost-effectiveness of Microsoft Academic Graph with machine learning for automated study identification in a living map of coronavirus disease 2019 (COVID-19) research.

Background: Identifying new, eligible studies for integration into living systematic reviews and maps usually relies on conventional Boolean updating searches of multiple databases and manual processing of the updated results. Automated searches of one, comprehensive, continuously updated source, with adjunctive machine learning, could enable more efficient searching, selection and prioritisation workflows for updating (living) reviews and maps, though research is needed to establish this. Microsoft Academic Graph (MAG) is a potentially comprehensive single source which also contains metadata that can be used in machine learning to help efficiently identify eligible studies. This study sought to establish whether: (a) MAG was a sufficiently sensitive single source to maintain our living map of COVID-19 research; and (b) eligible records could be identified with an acceptably high level of specificity. Methods: We conducted an eight-arm cost-effectiveness analysis to assess the costs, recall and precision of semi-automated workflows, incorporating MAG with adjunctive machine learning, for continually updating our living map. Resource use data (time use) were collected from information specialists and other researchers involved in map production. Our systematic review software, EPPI-Reviewer, was adapted to incorporate MAG and associated machine learning workflows, and also used to collect data on recall, precision, and manual screening workload. Results: The semi-automated MAG-enabled workflow dominated conventional workflows in both the base case and sensitivity analyses. At one month our MAG-enabled workflow with machine learning, active learning and fixed screening targets identified 469 additional, eligible articles for inclusion in our living map, and cost £3,179 GBP per week less, compared with conventional methods relying on Boolean searches of Medline and Embase. Conclusions: We were able to increase recall and coverage of a large living map, whilst reducing its production costs. This finding is likely to be transferrable to OpenAlex, MAG's successor database platform.

GRADE Guidelines 30: the GRADE approach to assessing the certainty of modeled evidence-An overview in the context of health decision-making.

OBJECTIVES: The objective of the study is to present the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) conceptual approach to the assessment of certainty of evidence from modeling studies (i.e., certainty associated with model outputs). STUDY DESIGN AND SETTING: Expert consultations and an international multidisciplinary workshop informed development of a conceptual approach to assessing the certainty of evidence from models within the context of systematic reviews, health technology assessments, and health care decisions. The discussions also clarified selected concepts and terminology used in the GRADE approach and by the modeling community. Feedback from experts in a broad range of modeling and health care disciplines addressed the content validity of the approach. RESULTS: Workshop participants agreed that the domains determining the certainty of evidence previously identified in the GRADE approach (risk of bias, indirectness, inconsistency, imprecision, reporting bias, magnitude of an effect, dose-response relation, and the direction of residual confounding) also apply when assessing the certainty of evidence from models. The assessment depends on the nature of model inputs and the model itself and on whether one is evaluating evidence from a single model or multiple models. We propose a framework for selecting the best available evidence from models: 1) developing de novo, a model specific to the situation of interest, 2) identifying an existing model, the outputs of which provide the highest certainty evidence for the situation of interest, either "off-the-shelf" or after adaptation, and 3) using outputs from multiple models. We also present a summary of preferred terminology to facilitate communication among modeling and health care disciplines. CONCLUSION: This conceptual GRADE approach provides a framework for using evidence from models in health decision-making and the assessment of certainty of evidence from a model or models. The GRADE Working Group and the modeling community are currently developing the detailed methods and related guidance for assessing specific domains determining the certainty of evidence from models across health care-related disciplines (e.g., therapeutic decision-making, toxicology, environmental health, and health economics).

Can choices between alternative hip prostheses be evidence based? a review of the economic evaluation literature.

BACKGROUND: Total hip replacement surgery places a considerable financial burden on health services and society. Given the large number of hip prostheses available to surgeons, reliable economic evidence is crucial to inform resource allocation decisions. This review summarises published economic evidence on alternative hip prostheses to examine the potential for the literature to inform resource allocation decisions in the UK. METHODS: We searched nine medical and economics electronic databases. 3,270 studies were initially identified, 17 studies were included in the review. Studies were critically appraised using three separate guidelines. RESULTS: Several methodological problems were identified including a lack of observed long term prosthesis survival data, limited up-to-date and UK based evidence and exclusion of patient and societal perspectives. CONCLUSIONS: More clinical trials including long term follow-up and economic evaluation are needed. These should compare the cost-effectiveness of different prostheses with longer-term follow-up and including a wider perspective.

Bone morphogenetic protein (BMP) for fracture healing in adults.

BACKGROUND: Delay in fracture healing is a complex clinical and economic issue for patients and health services. OBJECTIVES: To assess the incremental effectiveness and costs of bone morphogenetic protein (BMP) on fracture healing in acute fractures and nonunions compared with standards of care. SEARCH STRATEGY: We searched The Cochrane Library (2008, Issue 4), MEDLINE, and other major health and health economics databases (to October 2008). SELECTION CRITERIA: Randomised controlled trials (RCTs) and full or partial economic evaluations of BMP for fracture healing in skeletally mature adults. DATA COLLECTION AND ANALYSIS: All clinical and economic data were extracted by one author and checked by another. MAIN RESULTS: Eleven RCTs, all at high risk of bias, and four economic evaluations were included. Apart from one study, the times to fracture healing were comparable between the BMP and control groups. There was some evidence for increased healing rates, without requiring a secondary procedure, of BMP compared with usual care control in acute, mainly open, tibial fractures (risk ratio (RR) 1.19, 95% CI 0.99 to 1.43). The pooled RR for achieving union for nonunited fractures was 1.02 (95% CI 0.90 to 1.15). One study found no difference in union for patients who had corrective osteotomy for radial malunions. Data from three RCTs indicated that fewer secondary procedures were required for acute fracture patients treated with BMP versus controls (RR 0.65, 95% CI 0.50 to 0.83). Adverse events experienced were infection, hardware failure, pain, donor site morbidity, heterotopic bone formation and immunogenic reactions. The evidence on costs for BMP-2 for acute open tibia fractures is from one large RCT. This indicates that the direct medical costs associated with BMP would generally be higher than treatment with standard care, but this cost difference may decrease as fracture severity increases. Limited evidence suggests that the direct medical costs associated with BMP could be offset by faster healing and reduced time off work for patients with the most severe open tibia fractures. AUTHORS' CONCLUSIONS: This review highlights a paucity of data on the use of BMP in fracture healing as well as considerable industry involvement in currently available evidence. There is limited evidence to suggest that BMP may be more effective than controls for acute tibial fracture healing, however, the use of BMP for treating nonunion remains unclear. The limited available economic evidence indicates that BMP treatment for acute open tibial fractures may be more favourable economically when used in patients with the most severe fractures.