Psychosocial interventions for supporting women to stop smoking in pregnancy.

BACKGROUND: Tobacco smoking remains one of the few preventable factors associated with complications in pregnancy, and has serious long-term implications for women and babies. Smoking in pregnancy is decreasing in high-income countries, but is strongly associated with poverty and is increasing in low- to middle-income countries. OBJECTIVES: To assess the effects of smoking cessation interventions during pregnancy on smoking behaviour and perinatal health outcomes. SEARCH METHODS: In this sixth update, we searched the Cochrane Pregnancy and Childbirth Group's Trials Register (13 November 2015), checked reference lists of retrieved studies and contacted trial authors. SELECTION CRITERIA: Randomised controlled trials, cluster-randomised trials, and quasi-randomised controlled trials of psychosocial smoking cessation interventions during pregnancy. DATA COLLECTION AND ANALYSIS: Two review authors independently assessed trials for inclusion and trial quality, and extracted data. Direct comparisons were conducted in RevMan, with meta-regression conducted in STATA 14. MAIN RESULTS: The overall quality of evidence was moderate to high, with reductions in confidence due to imprecision and heterogeneity for some outcomes. One hundred and two trials with 120 intervention arms (studies) were included, with 88 trials (involving over 28,000 women) providing data on smoking abstinence in late pregnancy. Interventions were categorised as counselling, health education, feedback, incentives, social support, exercise and dissemination.In separate comparisons, there is high-quality evidence that counselling increased smoking cessation in late pregnancy compared with usual care (30 studies; average risk ratio (RR) 1.44, 95% confidence interval (CI) 1.19 to 1.73) and less intensive interventions (18 studies; average RR 1.25, 95% CI 1.07 to 1.47). There was uncertainty whether counselling increased the chance of smoking cessation when provided as one component of a broader maternal health intervention or comparing one type of counselling with another. In studies comparing counselling and usual care (largest comparison), it was unclear whether interventions prevented smoking relapse among women who had stopped smoking spontaneously in early pregnancy. However, a clear effect was seen in smoking abstinence at zero to five months postpartum (11 studies; average RR 1.59, 95% CI 1.26 to 2.01) and 12 to 17 months (two studies, average RR 2.20, 95% CI 1.23 to 3.96), with a borderline effect at six to 11 months (six studies; average RR 1.33, 95% CI 1.00 to 1.77). In other comparisons, the effect was unclear for most secondary outcomes, but sample sizes were small.Evidence suggests a borderline effect of health education compared with usual care (five studies; average RR 1.59, 95% CI 0.99 to 2.55), but the quality was downgraded to moderate as the effect was unclear when compared with less intensive interventions (four studies; average RR 1.20, 95% CI 0.85 to 1.70), alternative interventions (one study; RR 1.88, 95% CI 0.19 to 18.60), or when smoking cessation health education was provided as one component of a broader maternal health intervention.There was evidence feedback increased smoking cessation when compared with usual care and provided in conjunction with other strategies, such as counselling (average RR 4.39, 95% CI 1.89 to 10.21), but the confidence in the quality of evidence was downgraded to moderate as this was based on only two studies and the effect was uncertain when feedback was compared to less intensive interventions (three studies; average RR 1.29, 95% CI 0.75 to 2.20).High-quality evidence suggests incentive-based interventions are effective when compared with an alternative (non-contingent incentive) intervention (four studies; RR 2.36, 95% CI 1.36 to 4.09). However pooled effects were not calculable for comparisons with usual care or less intensive interventions (substantial heterogeneity, I2 = 93%).High-quality evidence suggests the effect is unclear in social support interventions provided by peers (six studies; average RR 1.42, 95% CI 0.98 to 2.07), in a single trial of support provided by partners, or when social support for smoking cessation was provided as part of a broader intervention to improve maternal health.The effect was unclear in single interventions of exercise compared to usual care (RR 1.20, 95% CI 0.72 to 2.01) and dissemination of counselling (RR 1.63, 95% CI 0.62 to 4.32).Importantly, high-quality evidence from pooled results demonstrated that women who received psychosocial interventions had a 17% reduction in infants born with low birthweight, a significantly higher mean birthweight (mean difference (MD) 55.60 g, 95% CI 29.82 to 81.38 g higher) and a 22% reduction in neonatal intensive care admissions. However the difference in preterm births and stillbirths was unclear. There did not appear to be adverse psychological effects from the interventions.The intensity of support women received in both the intervention and comparison groups has increased over time, with higher-intensity interventions more likely to have higher-intensity comparisons, potentially explaining why no clear differences were seen with increasing intervention intensity in meta-regression analyses. Among meta-regression analyses: studies classified as having 'unclear' implementation and unequal baseline characteristics were less effective than other studies. There was no clear difference between trials implemented by researchers (efficacy studies), and those implemented by routine pregnancy staff (effectiveness studies), however there was uncertainty in the effectiveness of counselling in four dissemination trials where the focus on the intervention was at an organisational level. The pooled effects were similar in interventions provided for women classified as having predominantly low socio-economic status, compared to other women. The effect was significant in interventions among women from ethnic minority groups; however not among indigenous women. There were similar effect sizes in trials with biochemically validated smoking abstinence and those with self-reported abstinence. It was unclear whether incorporating use of self-help manuals or telephone support increased the effectiveness of interventions. AUTHORS' CONCLUSIONS: Psychosocial interventions to support women to stop smoking in pregnancy can increase the proportion of women who stop smoking in late pregnancy and the proportion of infants born low birthweight. Counselling, feedback and incentives appear to be effective, however the characteristics and context of the interventions should be carefully considered. The effect of health education and social support is less clear. New trials have been published during the preparation of this review and will be included in the next update.

Narratives of community engagement: a systematic review-derived conceptual framework for public health interventions.

BACKGROUND: Government policy increasingly supports engaging communities to promote health. It is critical to consider whether such strategies are effective, for whom, and under what circumstances. However, 'community engagement' is defined in diverse ways and employed for different reasons. Considering the theory and context we developed a conceptual framework which informs understanding about what makes an effective (or ineffective) community engagement intervention. METHODS: We conducted a systematic review of community engagement in public health interventions using: stakeholder involvement; searching, screening, appraisal and coding of research literature; and iterative thematic syntheses and meta-analysis. A conceptual framework of community engagement was refined, following interactions between the framework and each review stage. RESULTS: From 335 included reports, three products emerged: (1) two strong theoretical 'meta-narratives': one, concerning the theory and practice of empowerment/engagement as an independent objective; and a more utilitarian perspective optimally configuring health services to achieve defined outcomes. These informed (2) models that were operationalized in subsequent meta-analysis. Both refined (3) the final conceptual framework. This identified multiple dimensions by which community engagement interventions may differ. Diverse combinations of intervention purpose, theory and implementation were noted, including: ways of defining communities and health needs; initial motivations for community engagement; types of participation; conditions and actions necessary for engagement; and potential issues influencing impact. Some dimensions consistently co-occurred, leading to three overarching models of effective engagement which either: utilised peer-led delivery; employed varying degrees of collaboration between communities and health services; or built on empowerment philosophies. CONCLUSIONS: Our conceptual framework and models are useful tools for considering appropriate and effective approaches to community engagement. These should be tested and adapted to facilitate intervention design and evaluation. Using this framework may disentangle the relative effectiveness of different models of community engagement, promoting effective, sustainable and appropriate initiatives.

The effectiveness of community engagement in public health interventions for disadvantaged groups: a meta-analysis.

BACKGROUND: Inequalities in health are acknowledged in many developed countries, whereby disadvantaged groups systematically suffer from worse health outcomes such as lower life expectancy than non-disadvantaged groups. Engaging members of disadvantaged communities in public health initiatives has been suggested as a way to reduce health inequities. This systematic review was conducted to evaluate the effectiveness of public health interventions that engage the community on a range of health outcomes across diverse health issues. METHODS: We searched the following sources for systematic reviews of public health interventions: Cochrane CDSR and CENTRAL, Campbell Library, DARE, NIHR HTA programme website, HTA database, and DoPHER. Through the identified reviews, we collated a database of primary studies that appeared to be relevant, and screened the full-text documents of those primary studies against our inclusion criteria. In parallel, we searched the NHS EED and TRoPHI databases for additional primary studies. For the purposes of these analyses, study design was limited to randomised and non-randomised controlled trials. Only interventions conducted in OECD countries and published since 1990 were included. We conducted a random effects meta-analysis of health behaviour, health consequences, self-efficacy, and social support outcomes, and a narrative summary of community outcomes. We tested a range of moderator variables, with a particular emphasis on the model of community engagement used as a potential moderator of intervention effectiveness. RESULTS: Of the 9,467 primary studies scanned, we identified 131 for inclusion in the meta-analysis. The overall effect size for health behaviour outcomes is d = .33 (95% CI .26, .40). The interventions were also effective in increasing health consequences (d = .16, 95% CI .06, .27); health behaviour self-efficacy (d = .41, 95% CI .16, .65) and perceived social support (d = .41, 95% CI .23, .65). Although the type of community engagement was not a significant moderator of effect, we identified some trends across studies. CONCLUSIONS: There is solid evidence that community engagement interventions have a positive impact on a range of health outcomes across various conditions. There is insufficient evidence to determine whether one particular model of community engagement is more effective than any other.

Reducing systematic review workload through certainty-based screening.

In systematic reviews, the growing number of published studies imposes a significant screening workload on reviewers. Active learning is a promising approach to reduce the workload by automating some of the screening decisions, but it has been evaluated for a limited number of disciplines. The suitability of applying active learning to complex topics in disciplines such as social science has not been studied, and the selection of useful criteria and enhancements to address the data imbalance problem in systematic reviews remains an open problem. We applied active learning with two criteria (certainty and uncertainty) and several enhancements in both clinical medicine and social science (specifically, public health) areas, and compared the results in both. The results show that the certainty criterion is useful for finding relevant documents, and weighting positive instances is promising to overcome the data imbalance problem in both data sets. Latent dirichlet allocation (LDA) is also shown to be promising when little manually-assigned information is available. Active learning is effective in complex topics, although its efficiency is limited due to the difficulties in text classification. The most promising criterion and weighting method are the same regardless of the review topic, and unsupervised techniques like LDA have a possibility to boost the performance of active learning without manual annotation.

The 'active ingredients' for successful community engagement with disadvantaged expectant and new mothers: a qualitative comparative analysis.

AIMS: To explore which conditions of community engagement are implicated in effective interventions targeting disadvantaged pregnant women and new mothers. BACKGROUND: Adaptive experiences during pregnancy and the early years are key to reducing health inequalities in women and children worldwide. Public health nurses, health visitors and community midwives are well placed to address such disadvantage, often using community engagement strategies. Such interventions are complex; however, and we need to better understand which aspects of community engagement are aligned with effectiveness. DESIGN: Qualitative comparative analysis conducted in 2013, of trials data included in a recently published systematic review. METHODS: Two reviewers agreed on relevant conditions from 24 maternity or early years intervention studies examining four models of community engagement. Effect size estimates were converted into 'fuzzy' effectiveness categories and truth tables were constructed. Using fsQCA software, Boolean minimization identified solution sets. Random effects multiple regression and fsQCA were conducted to rule out risk of methodological bias. RESULTS/FINDINGS: Studies focused on antenatal, immunization, breastfeeding and early professional intervention outcomes. Peer delivery (consistency 0·83; unique coverage 0·63); and mother-professional collaboration (consistency 0·833; unique coverage 0·21) were moderately aligned with effective interventions. Community-identified health need plus consultation/collaboration in intervention design and leading on delivery were weakly aligned with 'not effective' interventions (consistency 0·78; unique coverage 0·29). CONCLUSIONS: For disadvantaged new and expectant mothers, peer or collaborative delivery models could be used in interventions. A need exists to design and test community engagement interventions in other areas of maternity and early years care and to further evaluate models of empowerment.

Psychosocial interventions for supporting women to stop smoking in pregnancy.

BACKGROUND: Tobacco smoking in pregnancy remains one of the few preventable factors associated with complications in pregnancy, stillbirth, low birthweight and preterm birth and has serious long-term implications for women and babies. Smoking in pregnancy is decreasing in high-income countries, but is strongly associated with poverty and increasing in low- to middle-income countries. OBJECTIVES: To assess the effects of smoking cessation interventions during pregnancy on smoking behaviour and perinatal health outcomes. SEARCH METHODS: In this fifth update, we searched the Cochrane Pregnancy and Childbirth Group's Trials Register (1 March 2013), checked reference lists of retrieved studies and contacted trial authors to locate additional unpublished data. SELECTION CRITERIA: Randomised controlled trials, cluster-randomised trials, randomised cross-over trials, and quasi-randomised controlled trials (with allocation by maternal birth date or hospital record number) of psychosocial smoking cessation interventions during pregnancy. DATA COLLECTION AND ANALYSIS: Two review authors independently assessed trials for inclusion and trial quality, and extracted data. Direct comparisons were conducted in RevMan, and subgroup analyses and sensitivity analysis were conducted in SPSS. MAIN RESULTS: Eighty-six trials were included in this updated review, with 77 trials (involving over 29,000 women) providing data on smoking abstinence in late pregnancy.In separate comparisons, counselling interventions demonstrated a significant effect compared with usual care (27 studies; average risk ratio (RR) 1.44, 95% confidence interval (CI) 1.19 to 1.75), and a borderline effect compared with less intensive interventions (16 studies; average RR 1.35, 95% CI 1.00 to 1.82). However, a significant effect was only seen in subsets where counselling was provided in conjunction with other strategies. It was unclear whether any type of counselling strategy is more effective than others (one study; RR 1.15, 95% CI 0.86 to 1.53). In studies comparing counselling and usual care (the largest comparison), it was unclear whether interventions prevented smoking relapse among women who had stopped smoking spontaneously in early pregnancy (eight studies; average RR 1.06, 95% CI 0.93 to 1.21). However, a clear effect was seen in smoking abstinence at zero to five months postpartum (10 studies; average RR 1.76, 95% CI 1.05 to 2.95), a borderline effect at six to 11 months (six studies; average RR 1.33, 95% CI 1.00 to 1.77), and a significant effect at 12 to 17 months (two studies, average RR 2.20, 95% CI 1.23 to 3.96), but not in the longer term. In other comparisons, the effect was not significantly different from the null effect for most secondary outcomes, but sample sizes were small.Incentive-based interventions had the largest effect size compared with a less intensive intervention (one study; RR 3.64, 95% CI 1.84 to 7.23) and an alternative intervention (one study; RR 4.05, 95% CI 1.48 to 11.11).Feedback interventions demonstrated a significant effect only when compared with usual care and provided in conjunction with other strategies, such as counselling (two studies; average RR 4.39, 95% CI 1.89 to 10.21), but the effect was unclear when compared with a less intensive intervention (two studies; average RR 1.19, 95% CI 0.45 to 3.12).The effect of health education was unclear when compared with usual care (three studies; average RR 1.51, 95% CI 0.64 to 3.59) or less intensive interventions (two studies; average RR 1.50, 95% CI 0.97 to 2.31).Social support interventions appeared effective when provided by peers (five studies; average RR 1.49, 95% CI 1.01 to 2.19), but the effect was unclear in a single trial of support provided by partners.The effects were mixed where the smoking interventions were provided as part of broader interventions to improve maternal health, rather than targeted smoking cessation interventions.Subgroup analyses on primary outcome for all studies showed the intensity of interventions and comparisons has increased over time, with higher intensity interventions more likely to have higher intensity comparisons. While there was no significant difference, trials where the comparison group received usual care had the largest pooled effect size (37 studies; average RR 1.34, 95% CI 1.25 to 1.44), with lower effect sizes when the comparison group received less intensive interventions (30 studies; average RR 1.20, 95% CI 1.08 to 1.31), or alternative interventions (two studies; average RR 1.26, 95% CI 0.98 to 1.53). More recent studies included in this update had a lower effect size (20 studies; average RR 1.26, 95% CI 1.00 to 1.59), I(2)= 3%, compared to those in the previous version of the review (50 studies; average RR 1.50, 95% CI 1.30 to 1.73). There were similar effect sizes in trials with biochemically validated smoking abstinence (49 studies; average RR 1.43, 95% CI 1.22 to 1.67) and those with self-reported abstinence (20 studies; average RR 1.48, 95% CI 1.17 to 1.87). There was no significant difference between trials implemented by researchers (efficacy studies), and those implemented by routine pregnancy staff (effectiveness studies), however the effect was unclear in three dissemination trials of counselling interventions where the focus on the intervention was at an organisational level (average RR 0.96, 95% CI 0.37 to 2.50). The pooled effects were similar in interventions provided for women with predominantly low socio-economic status (44 studies; average RR 1.41, 95% CI 1.19 to 1.66), compared to other women (26 studies; average RR 1.47, 95% CI 1.21 to 1.79); though the effect was unclear in interventions among women from ethnic minority groups (five studies; average RR 1.08, 95% CI 0.83 to 1.40) and aboriginal women (two studies; average RR 0.40, 95% CI 0.06 to 2.67). Importantly, pooled results demonstrated that women who received psychosocial interventions had an 18% reduction in preterm births (14 studies; average RR 0.82, 95% CI 0.70 to 0.96), and infants born with low birthweight (14 studies; average RR 0.82, 95% CI 0.71 to 0.94). There did not appear to be any adverse effects from the psychosocial interventions, and three studies measured an improvement in women's psychological wellbeing. AUTHORS' CONCLUSIONS: Psychosocial interventions to support women to stop smoking in pregnancy can increase the proportion of women who stop smoking in late pregnancy, and reduce low birthweight and preterm births.

Using machine learning to extract information and predict outcomes from reports of randomised trials of smoking cessation interventions in the Human Behaviour-Change Project.

Background  Using reports of randomised trials of smoking cessation interventions as a test case, this study aimed to develop and evaluate machine learning (ML) algorithms for extracting information from study reports and predicting outcomes as part of the Human Behaviour-Change Project. It is the first of two linked papers, with the second paper reporting on further development of a prediction system. Methods  Researchers manually annotated 70 items of information ('entities') in 512 reports of randomised trials of smoking cessation interventions covering intervention content and delivery, population, setting, outcome and study methodology using the Behaviour Change Intervention Ontology. These entities were used to train ML algorithms to extract the information automatically. The information extraction ML algorithm involved a named-entity recognition system using the 'FLAIR' framework. The manually annotated intervention, population, setting and study entities were used to develop a deep-learning algorithm using multiple layers of long-short-term-memory (LSTM) components to predict smoking cessation outcomes. Results  The F1 evaluation score, derived from the false positive and false negative rates (range 0-1), for the information extraction algorithm averaged 0.42 across different types of entity (SD=0.22, range 0.05-0.88) compared with an average human annotator's score of 0.75 (SD=0.15, range 0.38-1.00). The algorithm for assigning entities to study arms ( e.g., intervention or control) was not successful. This initial ML outcome prediction algorithm did not outperform prediction based just on the mean outcome value or a linear regression model. Conclusions  While some success was achieved in using ML to extract information from reports of randomised trials of smoking cessation interventions, we identified major challenges that could be addressed by greater standardisation in the way that studies are reported. Outcome prediction from smoking cessation studies may benefit from development of novel algorithms, e.g., using ontological information to inform ML (as reported in the linked paper 3).

Using systems perspectives in evidence synthesis: A methodological mapping review.

Reviewing complex interventions is challenging because they include many elements that can interact dynamically in a nonlinear manner. A systems perspective offers a way of thinking to help understand complex issues, but its application in evidence synthesis is not established. The aim of this project was to understand how and why systems perspectives have been applied in evidence synthesis. A methodological mapping review was conducted to identify papers using a systems perspective in evidence synthesis. A search was conducted in seven bibliographic databases and three search engines. A total of 101 papers (representing 98 reviews) met the eligibility criteria. Two categories of reviews were identified: (1) reviews using a "systems lens" to frame the topic, generate hypotheses, select studies, and guide the analysis and interpretation of findings (n = 76) and (2) reviews using systems methods to develop a systems model (n = 22). Several methods (e.g., systems dynamic modeling, soft systems approach) were identified, and they were used to identify, rank and select elements, analyze interactions, develop models, and forecast needs. The main reasons for using a systems perspective were to address complexity, view the problem as a whole, and understand the interrelationships between the elements. Several challenges for capturing the true nature and complexity of a problem were raised when performing these methods. This review is a useful starting point when designing evidence synthesis of complex interventions. It identifies different opportunities for applying a systems perspective in evidence synthesis, and highlights both commonplace and less familiar methods.

School closure in response to epidemic outbreaks: Systems-based logic model of downstream impacts.

Background: School closures have been a recommended non-pharmaceutical intervention in pandemic response owing to the potential to reduce transmission of infection between children, school staff and those that they contact. However, given the many roles that schools play in society, closure for any extended period is likely to have additional impacts. Literature reviews of research exploring school closure to date have focused upon epidemiological effects; there is an unmet need for research that considers the multiplicity of potential impacts of school closures. Methods: We used systematic searching, coding and synthesis techniques to develop a systems-based logic model. We included literature related to school closure planned in response to epidemics large and small, spanning the 1918-19 'flu pandemic through to the emerging literature on the 2019 novel coronavirus. We used over 170 research studies and a number of policy documents to inform our model. Results: The model organises the concepts used by authors into seven higher level domains: children's health and wellbeing, children's education, impacts on teachers and other school staff, the school organisation, considerations for parents and families, public health considerations, and broader economic impacts. The model also collates ideas about potential moderating factors and ethical considerations. While dependent upon the nature of epidemics experienced to date, we aim for the model to provide a starting point for theorising about school closures in general, and as part of a wider system that is influenced by contextual and population factors. Conclusions: The model highlights that the impacts of school closures are much broader than those related solely to health, and demonstrates that there is a need for further concerted work in this area. The publication of this logic model should help to frame future research in this area and aid decision-makers when considering future school closure policy and possible mitigation strategies.

Handsearching had best recall but poor efficiency when exporting to a bibliographic tool: case study.

OBJECTIVES: The objective of this study was to compare the effectiveness and efficiency of methods used to identify and export conference abstracts into a bibliographic management tool. STUDY DESIGN AND SETTING: This is a case study. The effectiveness and efficiency of methods to identify and export conference abstracts presented at the American Society of Hematology (ASH) conference 2016-2018 for a systematic review were evaluated. A reference standard handsearch of conference proceedings was compared with: 1) contacting Blood (the journal that report ASH proceedings); 2) keyword searching; 3) searching Embase; 4) searching MEDLINE via EndNote; and 5) searching CPCI-S. Effectiveness was determined by the number of abstracts identified compared with the reference standard, whereas efficiency was a comparison between the resources required to identify and export conference abstracts compared with the reference standard. RESULTS: Six hundred and four potentially eligible and 15 confirmed eligible conference abstracts (abstracts included in the review) were identified by the handsearch. Comparator 2 was the only method to identify all abstracts and it was more efficient than the reference standard. Comparators 1 and 3-5 missed a number of eligible abstracts. CONCLUSION: This study raises potentially concerning questions about searching for conferences' abstracts by methods other than directly searching the original conference proceedings. Efficiency of exporting would be improved if journals permitted bulk downloads.

The Human Behaviour-Change Project: An artificial intelligence system to answer questions about changing behaviour.

Changing behaviour is necessary to address many of the threats facing human populations.  However, identifying behaviour change interventions likely to be effective in particular contexts as a basis for improving them presents a major challenge. The Human Behaviour-Change Project harnesses the power of artificial intelligence and behavioural science to organise global evidence about behaviour change to predict outcomes in common and unknown behaviour change scenarios.

How can additional secondary data analysis of observational data enhance the generalisability of meta-analytic evidence for local public health decision making?

This paper critically explores how survey and routinely collected data could aid in assessing the generalisability of public health evidence. We propose developing approaches that could be employed in understanding the relevance of public health evidence, and investigate ways of producing meta-analytic estimates tailored to reflect local circumstances, based on analyses of secondary data. Currently, public health decision makers face challenges in interpreting global review evidence to assess its meaning in local contexts. A lack of clarity on the definition and scope of generalisability, and the absence of consensus on its measurement, has stunted methodological progress. The consequence of failing to tackle generalisability means that systematic review evidence often fails to fulfil its potential contribution in public health decision making. Three approaches to address these problems are considered and emerging challenges discussed: (1) purposeful exploration after a review has been conducted, and we present a framework of potential avenues of enquiry and a worked example; (2) recalibration of the results to weight studies differentially based on their similarity to conditions in an inference population, and we provide a worked example using UK Census data to understand potential differences in the effectiveness of community engagement interventions among sites in England and Wales; (3) purposeful exploration before starting a review to ensure that the findings are relevant to an inference population. The paper aims to demonstrate how a more nuanced treatment of context in reviews of public health interventions could be achieved through greater engagement with existing large sources of secondary data.

The development and evaluation of an online application to assist in the extraction of data from graphs for use in systematic reviews.

Background: The extraction of data from the reports of primary studies, on which the results of systematic reviews depend, needs to be carried out accurately. To aid reliability, it is recommended that two researchers carry out data extraction independently. The extraction of statistical data from graphs in PDF files is particularly challenging, as the process is usually completely manual, and reviewers need sometimes to revert to holding a ruler against the page to read off values: an inherently time-consuming and error-prone process. Methods: To mitigate some of the above problems we integrated and customised two existing JavaScript libraries to create a new web-based graphical data extraction tool to assist reviewers in extracting data from graphs. This tool aims to facilitate more accurate and timely data extraction through a user interface which can be used to extract data through mouse clicks. We carried out a non-inferiority evaluation to examine its performance in comparison with participants' standard practice for extracting data from graphs in PDF documents. Results: We found that the customised graphical data extraction tool is not inferior to users' (N=10) prior standard practice. Our study was not designed to show superiority, but suggests that, on average, participants saved around 6 minutes per graph using the new tool, accompanied by a substantial increase in accuracy. Conclusions: Our study suggests that the incorporation of this type of tool in online systematic review software would be beneficial in facilitating the production of accurate and timely evidence synthesis to improve decision-making.

Just how plain are plain tobacco packs: re-analysis of a systematic review using multilevel meta-analysis suggests lessons about the comparative benefits of synthesis methods.

BACKGROUND: Comparisons between narrative synthesis and meta-analysis as synthesis methods in systematic reviews are uncommon within the same systematic review. We re-analysed a systematic review on the effects of plain packaging of tobacco on attractiveness. We sought to compare different synthesis approaches within the same systematic review and shed light on the comparative benefits of each approach. METHODS: In our re-analysis, we included results relating to attractiveness in included reports. We extracted findings from studies and converted all estimates of differences in attractiveness to Cohen's d. We used multilevel meta-analysis to account for clustering of effect sizes within studies. RESULTS: Of the 19 studies reporting results on attractiveness, seven studies that included between-subjects analyses could be included in the meta-analysis. Plain packs were less attractive than branded packs (d = - 0.59, 95% CI [- 0.71, - 0.47]), with negligible but uncertain between-studies heterogeneity (I2 = 0%, 95% CI [0.00, 70.81]) and high within-study heterogeneity (I2 = 92.6%, 95% CI [91.04, 93.90]). CONCLUSIONS: The meta-analysis found, similar to the narrative synthesis, that respondents typically rated plain packaging as less attractive than alternative (e.g. branded) tobacco packs. However, there were several trade-offs between analysis methods in the types and bodies of evidence each one contained and in the difference between partial precision and breadth of conclusions. Analysis methods were different in respect of the role of judgement and contextual variation and in terms of estimation and unexpected effect modification. In addition, we noted that analysis methods were different in how they accounted for heterogeneity and consistency.

Text mining for search term development in systematic reviewing: A discussion of some methods and challenges.

Using text mining to aid the development of database search strings for topics described by diverse terminology has potential benefits for systematic reviews; however, methods and tools for accomplishing this are poorly covered in the research methods literature. We briefly review the literature on applications of text mining for search term development for systematic reviewing. We found that the tools can be used in 5 overarching ways: improving the precision of searches; identifying search terms to improve search sensitivity; aiding the translation of search strategies across databases; searching and screening within an integrated system; and developing objectively derived search strategies. Using a case study and selected examples, we then reflect on the utility of certain technologies (term frequency-inverse document frequency and Termine, term frequency, and clustering) in improving the precision and sensitivity of searches. Challenges in using these tools are discussed. The utility of these tools is influenced by the different capabilities of the tools, the way the tools are used, and the text that is analysed. Increased awareness of how the tools perform facilitates the further development of methods for their use in systematic reviews.

Using the realist perspective to link theory from qualitative evidence synthesis to quantitative studies: Broadening the matrix approach.

BACKGROUND: This study describes an approach for the use of a specific type of qualitative evidence synthesis in the matrix approach, a mixed studies reviewing method. The matrix approach compares quantitative and qualitative data on the review level by juxtaposing concrete recommendations from the qualitative evidence synthesis against interventions in primary quantitative studies. However, types of qualitative evidence syntheses that are associated with theory building generate theoretical models instead of recommendations. Therefore, the output from these types of qualitative evidence syntheses cannot directly be used for the matrix approach but requires transformation. This approach allows for the transformation of these types of output. METHOD: The approach enables the inference of moderation effects instead of direct effects from the theoretical model developed in a qualitative evidence synthesis. Recommendations for practice are formulated on the basis of interactional relations inferred from the qualitative evidence synthesis. In doing so, we apply the realist perspective to model variables from the qualitative evidence synthesis according to the context-mechanism-outcome configuration. FINDINGS: A worked example shows that it is possible to identify recommendations from a theory-building qualitative evidence synthesis using the realist perspective. We created subsets of the interventions from primary quantitative studies based on whether they matched the recommendations or not and compared the weighted mean effect sizes of the subsets. The comparison shows a slight difference in effect sizes between the groups of studies. The study concludes that the approach enhances the applicability of the matrix approach.

The Human Behaviour-Change Project: harnessing the power of artificial intelligence and machine learning for evidence synthesis and interpretation.

BACKGROUND: Behaviour change is key to addressing both the challenges facing human health and wellbeing and to promoting the uptake of research findings in health policy and practice. We need to make better use of the vast amount of accumulating evidence from behaviour change intervention (BCI) evaluations and promote the uptake of that evidence into a wide range of contexts. The scale and complexity of the task of synthesising and interpreting this evidence, and increasing evidence timeliness and accessibility, will require increased computer support. The Human Behaviour-Change Project (HBCP) will use Artificial Intelligence and Machine Learning to (i) develop and evaluate a 'Knowledge System' that automatically extracts, synthesises and interprets findings from BCI evaluation reports to generate new insights about behaviour change and improve prediction of intervention effectiveness and (ii) allow users, such as practitioners, policy makers and researchers, to easily and efficiently query the system to get answers to variants of the question 'What works, compared with what, how well, with what exposure, with what behaviours (for how long), for whom, in what settings and why?'. METHODS: The HBCP will: a) develop an ontology of BCI evaluations and their reports linking effect sizes for given target behaviours with intervention content and delivery and mechanisms of action, as moderated by exposure, populations and settings; b) develop and train an automated feature extraction system to annotate BCI evaluation reports using this ontology; c) develop and train machine learning and reasoning algorithms to use the annotated BCI evaluation reports to predict effect sizes for particular combinations of behaviours, interventions, populations and settings; d) build user and machine interfaces for interrogating and updating the knowledge base; and e) evaluate all the above in terms of performance and utility. DISCUSSION: The HBCP aims to revolutionise our ability to synthesise, interpret and deliver evidence on behaviour change interventions that is up-to-date and tailored to user need and context. This will enhance the usefulness, and support the implementation of, that evidence.

Meta-analysis, complexity, and heterogeneity: a qualitative interview study of researchers' methodological values and practices.

BACKGROUND: Complex or heterogeneous data pose challenges for systematic review and meta-analysis. In recent years, a number of new methods have been developed to meet these challenges. This qualitative interview study aimed to understand researchers' understanding of complexity and heterogeneity and the factors which may influence the choices researchers make in synthesising complex data. METHODS: We conducted interviews with a purposive sample of researchers (N = 19) working in systematic review or meta-analysis across a range of disciplines. We analysed data thematically using a framework approach. RESULTS: Participants reported using a broader range of methods and data types in complex reviews than in traditional reviews. A range of techniques are used to explore heterogeneity, but there is some debate about their validity, particularly when applied post hoc. CONCLUSIONS: Technical considerations of how to synthesise complex evidence cannot be isolated from questions of the goals and contexts of research. However, decisions about how to analyse data appear to be made in a largely informal way, drawing on tacit expertise, and their relation to these broader questions remains unclear.

Using text mining for study identification in systematic reviews: a systematic review of current approaches.

BACKGROUND: The large and growing number of published studies, and their increasing rate of publication, makes the task of identifying relevant studies in an unbiased way for inclusion in systematic reviews both complex and time consuming. Text mining has been offered as a potential solution: through automating some of the screening process, reviewer time can be saved. The evidence base around the use of text mining for screening has not yet been pulled together systematically; this systematic review fills that research gap. Focusing mainly on non-technical issues, the review aims to increase awareness of the potential of these technologies and promote further collaborative research between the computer science and systematic review communities. METHODS: Five research questions led our review: what is the state of the evidence base; how has workload reduction been evaluated; what are the purposes of semi-automation and how effective are they; how have key contextual problems of applying text mining to the systematic review field been addressed; and what challenges to implementation have emerged? We answered these questions using standard systematic review methods: systematic and exhaustive searching, quality-assured data extraction and a narrative synthesis to synthesise findings. RESULTS: The evidence base is active and diverse; there is almost no replication between studies or collaboration between research teams and, whilst it is difficult to establish any overall conclusions about best approaches, it is clear that efficiencies and reductions in workload are potentially achievable. On the whole, most suggested that a saving in workload of between 30% and 70% might be possible, though sometimes the saving in workload is accompanied by the loss of 5% of relevant studies (i.e. a 95% recall). CONCLUSIONS: Using text mining to prioritise the order in which items are screened should be considered safe and ready for use in 'live' reviews. The use of text mining as a 'second screener' may also be used cautiously. The use of text mining to eliminate studies automatically should be considered promising, but not yet fully proven. In highly technical/clinical areas, it may be used with a high degree of confidence; but more developmental and evaluative work is needed in other disciplines.

Using qualitative comparative analysis (QCA) in systematic reviews of complex interventions: a worked example.

BACKGROUND: Systematic reviews that address policy and practice questions in relation to complex interventions frequently need not only to assess the efficacy of a given intervention but to identify which intervention - and which intervention components - might be most effective in particular situations. Here, intervention replication is rare, and commonly used synthesis methods are less useful when the focus of analysis is the identification of those components of an intervention that are critical to its success. METHODS: Having identified initial theories of change in a previous analysis, we explore the potential of qualitative comparative analysis (QCA) to assist with complex syntheses through a worked example. Developed originally in the area of political science and historical sociology, a QCA aims to identify those configurations of participant, intervention and contextual characteristics that may be associated with a given outcome. Analysing studies in these terms facilitates the identification of necessary and sufficient conditions for the outcome to be obtained. Since QCA is predicated on the assumption that multiple pathways might lead to the same outcome and does not assume a linear additive model in terms of changes to a particular condition (that is, it can cope with 'tipping points' in complex interventions), it appears not to suffer from some of the limitations of the statistical methods often used in meta-analysis. RESULTS: The worked example shows how the QCA reveals that our initial theories of change were unable to distinguish between 'effective' and 'highly effective' interventions. Through the iterative QCA process, other intervention characteristics are identified that better explain the observed results. CONCLUSIONS: QCA is a promising alternative (or adjunct), particularly to the standard fall-back of a 'narrative synthesis' when a quantitative synthesis is impossible, and should be considered when reviews are broad and heterogeneity is significant. There are very few examples of its use with systematic review data at present, and further methodological work is needed to establish optimal conditions for its use and to document process, practice, and reporting standards.