Development of the informed health choices resources in four countries to teach primary school children to assess claims about treatment effects: a qualitative study employing a user-centred approach.

BACKGROUND: People of all ages are flooded with health claims about treatment effects (benefits and harms of treatments). Many of these are not reliable, and many people lack skills to assess their reliability. Primary school is the ideal time to begin to teach these skills, to lay a foundation for continued learning and enable children to make well-informed health choices, as they grow older. However, these skills are rarely being taught and yet there are no rigorously developed and evaluated resources for teaching these skills. OBJECTIVES: To develop the Informed Health Choices (IHC) resources (for learning and teaching people to assess claims about the effects of treatments) for primary school children and teachers. METHODS: We prototyped, piloted, and user-tested resources in four settings that included Uganda, Kenya, Rwanda, and Norway. We employed a user-centred approach to designing IHC resources which entailed multiple iterative cycles of development (determining content scope, generating ideas, prototyping, testing, analysing and refining) based on continuous close collaboration with teachers and children. RESULTS: We identified 24 Key Concepts that are important for children to learn. We developed a comic book and a separate exercise book to introduce and explain the Key Concepts to the children, combining lessons with exercises and classroom activities. We developed a teachers' guide to supplement the resources for children. CONCLUSION: By employing a user-centred approach to designing resources to teach primary children to think critically about treatment claims and choices, we developed learning resources that end users experienced as useful, easy to use and well-suited to use in diverse classroom settings.

Effects of the Informed Health Choices podcast on the ability of parents of primary school children in Uganda to assess the trustworthiness of claims about treatment effects: one-year follow up of a randomised trial.

INTRODUCTION: Earlier, we designed and evaluated an educational mass media intervention for improving people's ability to think more critically and to assess the trustworthiness of claims (assertions) about the benefits and harms (effects) of treatments. The overall aims of this follow-up study were to evaluate the impact of our intervention 1 year after it was administered, and to assess retention of learning and behaviour regarding claims about treatments. METHODS: We randomly allocated consenting parents to listen to either the Informed Health Choices podcast (intervention) or typical public service announcements about health issues (control) over 7-10 weeks. Each intervention episode explained how the trustworthiness of treatment claims can be assessed by using relevant key concepts of evidence-informed decision-making. Participants listened to two episodes per week, delivered by research assistants. We evaluated outcomes immediately, and a year after the intervention. Primary outcomes were mean score and the proportion with a score indicating a basic ability to apply the key concepts (> 11 out of 18 correct answers) on a tool measuring people's ability to critically appraise the trustworthiness of treatment claims. Skills decay/retention was estimated by calculating the relative difference between the follow-up and initial results in the intervention group, adjusting for chance. Statistical analyses were performed using R (R Core Team, Vienna, Austria; version 3.4.3). RESULTS: After 1 year, the mean score for parents in the intervention group was 58.9% correct answers, compared to 52.6% in the control (adjusted mean difference of 6.7% (95% CI 3.3% to 10.1%)). In the intervention group, 47.2% of 267 parents had a score indicating a basic ability to assess treatment claims compared to 39.5% of 256 parents in the control (adjusted difference of 9.8% more parents (95% CI 0.9% to 18.9%). These represent relative reductions of 29% in the mean scores and 33% in the proportion of parents with a score indicating a basic ability to assess the trustworthiness of claims about treatment effects. CONCLUSIONS: Although listening to the Informed Health Choices podcast initially led to a large improvement in the ability of parents to assess claims about the effects of treatments, our findings show that these skills decreased substantially over 1 year. More active practice could address the substantial skills decay observed over 1 year. TRIAL REGISTRATION: Pan African Clinical Trial Registry (www.pactr.org), PACTR201606001676150. Registered on 12 June 2016.

Effects of the Informed Health Choices primary school intervention on the ability of children in Uganda to assess the reliability of claims about treatment effects, 1-year follow-up: a cluster-randomised trial.

INTRODUCTION: We evaluated an intervention designed to teach 10- to 12-year-old primary school children to assess claims about the effects of treatments (any action intended to maintain or improve health). We report outcomes measured 1 year after the intervention. METHODS: In this cluster-randomised trial, we included primary schools in the central region of Uganda that taught year 5 children (aged 10 to 12 years). We randomly allocated a representative sample of eligible schools to either an intervention or control group. Intervention schools received the Informed Health Choices primary school resources (textbooks, exercise books and a teachers' guide). The primary outcomes, measured at the end of the school term and again after 1 year, were the mean score on a test with two multiple-choice questions for each of the 12 concepts and the proportion of children with passing scores. RESULTS: We assessed 2960 schools for eligibility; 2029 were eligible, and a random sample of 170 were invited to recruitment meetings. After recruitment meetings, 120 eligible schools consented and were randomly assigned to either the intervention group (n = 60 schools; 76 teachers and 6383 children) or the control group (n = 60 schools; 67 teachers and 4430 children). After 1 year, the mean score in the multiple-choice test for the intervention schools was 68.7% compared with 53.0% for the control schools (adjusted mean difference 16.7%; 95% CI, 13.9 to 19.5; P < 0.00001). In the intervention schools, 3160 (80.1%) of 3943 children who completed the test after 1 year achieved a predetermined passing score (≥ 13 of 24 correct answers) compared with 1464 (51.5%) of 2844 children in the control schools (adjusted difference, 39.5%; 95% CI, 29.9 to 47.5). CONCLUSION: Use of the learning resources led to a large improvement in the ability of children to assess claims, which was sustained for at least 1 year. TRIAL REGISTRATION: Pan African Clinical Trial Registry (www.pactr.org), PACTR201606001679337. Registered on 13 June 2016.

Informed Health Choices media intervention for improving people's ability to critically appraise the trustworthiness of claims about treatment effects: a mixed-methods process evaluation of a randomised trial in Uganda.

We developed the Informed Health Choices podcast to improve people's ability to assess claims about the effects of treatments. We evaluated the effects of the podcast in a randomised trial. OBJECTIVES: We conducted this process evaluation to assess the fidelity of the intervention, identify factors that affected the implementation and impact of the intervention and could affect scaling up, and identify potential adverse and beneficial effects. SETTING: The study was conducted in central Uganda in rural, periurban and urban settings. PARTICIPANTS: We collected data on parents who were in the intervention arm of the Informed Health Choices study that evaluated an intervention to improve parents' ability to assess treatment effects. PROCEDURES: We conducted 84 semistructured interviews during the intervention, 19 in-depth interviews shortly after, two focus group discussions with parents, one focus group discussion with research assistants and two in-depth interviews with the principal investigators. We used framework analysis to manage qualitative data, assessed the certainty of the findings using the GRADE-CERQual (Grading of Recommendations, Assessment, Development and Evaluations-Confidence in the Evidence from Reviews of Qualitative Research) approach, and organised findings in a logic model. OUTCOMES: Proportion of participants listening to all episodes; factors influencing the implementation of the podcast; ways to scale up and any adverse and beneficial effects. RESULTS: All participants who completed the study listened to the podcast as intended, perhaps because of the explanatory design and recruitment of parents with a positive attitude. This was also likely facilitated by the podcast being delivered by research assistants, and providing the participants with MP3 players. The podcast was reportedly clear, understandable, credible and entertaining, which motivated them to listen and eased implementation. No additional adverse effects were reported. CONCLUSIONS: Participants experienced the podcast positively and were motivated to engage with it. These findings help to explain the short-term effectiveness of the intervention, but not the decrease in effectiveness over the following year.

Evaluating people's ability to assess treatment claims: Validating a test in Mandarin from Claim Evaluation Tools database.

OBJECTIVE: To describe the psychometric testing using Rasch analysis of a test in Mandarin developed from the Claim Evaluation Tools database. METHODS: We translated selected MCQs from the IHC Claim Evaluation Tools database to Mandarin and created a test including 24 MCQs covering 11 key concepts. We used purposeful sampling and surveyed children and adults in the Lanzhou area. In total 389 responses were entered into the analysis. We evaluated the psychometric properties of the test using Rasch analysis and the RUMM2030 software, testing for internal construct validity (multidimensionality), invariance of the items (item-person interaction), and item bias (differential item functioning). RESULTS: Overall, the psychometric properties of the test were found to be satisfactory. Based on findings from the Rasch analysis, we deleted three MCQs with suboptimal fit. CONCLUSIONS: The resulting test includes 21 MCQs and can be used in school and other teaching settings, in randomized trials evaluating outcomes of educational interventions, or in cross-sectional studies in Mandarin-speaking populations in China.

The James Lind Initiative: books, websites and databases to promote critical thinking about treatment claims, 2003 to 2018.

BACKGROUND: The James Lind Initiative (JLI) was a work programme inaugurated by Iain Chalmers and Patricia Atkinson to press for better research for better health care. It ran between 2003 and 2018, when Iain Chalmers retired. During the 15 years of its existence, the JLI developed three strands of work in collaboration with the authors of this paper, and with others. WORK THEMES: The first work strand involved developing a process for use by patients, carers and clinicians to identify shared priorities for research - the James Lind Alliance. The second strand was a series of articles, meetings, prizes and other developments to raise awareness of the massive amounts of avoidable waste in research, and of ways of reducing it. The third strand involved using a variety of approaches to promote better public and professional understanding of the importance of research in clinical practice and public health. JLI work on the first two themes has been addressed in previously published reports. This paper summarises JLI involvement during the 15 years of its existence in giving talks, convening workshops, writing books, and creating websites and databases to promote critical thinking about treatment claims. CONCLUSION: During its 15-year life, the James Lind Initiative worked collaboratively with others to create free teaching and learning resources to help children and adults learn how to recognise untrustworthy claims about the effects of treatments. These resources have been translated in more than twenty languages, but much more could be done to support their uptake and wider use.

Development of mass media resources to improve the ability of parents of primary school children in Uganda to assess the trustworthiness of claims about the effects of treatments: a human-centred design approach.

BACKGROUND: Claims about what we need to do to improve our health are everywhere. Most interventions simply tell people what to do, and do not empower them to critically assess health information. Our objective was to design mass media resources to enable the public to critically appraise the trustworthiness of claims about the benefits and harms of treatments and make informed health choices. METHODS: Research was conducted between 2013 and 2016 across multiple iterative phases. Participants included researchers, journalists, parents, other members of the public. First, we developed a list of 32 key concepts that people need to understand to be able to assess the trustworthiness of claims about treatment effects. Next, we used a human-centred design approach, to generate ideas for resources for teaching the key concepts, and developed and user-tested prototypes through qualitative interviews. We addressed identified problems and repeated this process until we had a product that was deemed relevant and desirable by our target audience, and feasible to implement. RESULTS: We generated over 160 ideas, mostly radio-based. After prototyping some of these, we found that a podcast produced collaboratively by health researchers and journalists was the most promising approach. We developed eight episodes of the Informed Health Choices podcast, a song on critical thinking about treatments and a reminder checklist. Early versions of the podcast were reportedly too long, boring and confusing. We shortened the episodes, included one key concept per episode, and changed to story-telling with skits. The final version of the podcast was found to be useful, understandable, credible and desirable. CONCLUSION: We found many problems with various prototypes of mass media resources. Using a human-centred design approach, we overcame those problems. We have developed a guide to help others prepare similar podcasts.

Validación de un cuestionario para medir la habilidad de la población general para evaluar afirmaciones acerca de tratamientos médicos.

INTRODUCCIÓN: Todos los días, las personas se enfrentan a afirmaciones acerca de tratamientos en medios de comunicación, redes sociales o por viva voz. OBJETIVO: Validar un cuestionario en español para medir las habilidades de un individuo para evaluar afirmaciones acerca de tratamientos. MÉTODO: Veintidós preguntas de opción múltiple de la base de datos Claim Evaluation Tools fueron traducidas y aplicadas a 172 niños y 268 adultos. Mediante un modelo Rasch se exploró el ajuste promedio e individual por reactivo, el potencial comportamiento diferencial del reactivo (basado en el género, edad y modo de aplicación), la multidimensionalidad y la independencia local. RESULTADOS: El ajuste promedio por reactivo fue apropiado. Cuatro preguntas de opción múltiple mostraron pobre ajuste. La fiabilidad del cuestionario fue satisfactoria, con un índice de separación de 0.7. Las preguntas de opción múltiple fueron unidimensionales, y no hubo dependencia específica. CONCLUSIÓN: Se obtuvo un conjunto de 18 preguntas de opción múltiple con ajuste satisfactorio. El cuestionario es el primero disponible y validado en español para medir las habilidades de los individuos para evaluar afirmaciones acerca de tratamientos. INTRODUCTION: Every day, people are faced with claims about treatment effects through mass media, social media, or by word of mouth. OBJECTIVE: To validate a Spanish-language questionnaire to measure the ability of an individual to assess claims about treatments effects. METHOD: A set of 22 multiple choice questions taken from the claim evaluation tools database were translated and applied to 172 children and 268 adults. Using a Rasch model, overall and individual item-person fit was explored, as well as the potential item differential functioning (based on gender, age and mode of administration), multidimensionality and local independence. RESULTS: Overall item-person fit was appropriate. Four multiple-choice questions showed a poor fit. Reliability of the questionnaire was satisfactory with a person separation index of 0.7. Multiple-choice questions were unidimensional, and there was no specific dependency. CONCLUSION: A set of 18 multiple-choice questions with satisfactory fit was obtained. This is the first available questionnaire validated in Spanish to measure individuals' ability to assess claims about treatment effects.

"A waste of time without patients": The views of patient representatives attending a workshop in evidence-based practice.

BACKGROUND: Shared decision-making is a central element of evidence-based practice (EBP). Training in EBP has traditionally focused on providers, but there is an increasing interest in developing such educational resources for patients. The aim of this study is to explore the views of patient representatives attending a workshop in EBP. METHODS: We conducted three focus groups with participants attending EBP workshops in 2013, 2014, and 2015. We used the framework method for content analysis. We reviewed the transcribed interviews independently and assigned initial codes and final categories to the transcriptions. We created an analytical framework and a flow diagram to visualize the category hierarchy and the relationship between categories. RESULTS: We identified two main themes; "How to facilitate training in evidence-based practice for patients," and "Outputs of training in evidence based practice for patients." Training in EBP for patient representatives "should reflect the principles EBP," and include interaction with both health professionals and other representatives. The educational needs of representatives are much the same as those of health professionals, and the training should therefore be "on a par with professionals." The relevance of such training may depend on the representatives' mandate, and costs might be an important barrier. Important outputs of such training include "becoming a knowledge manager," "enabling participation and informed decisions" and "re-defining the patient representatives' role." CONCLUSIONS: The findings of this study suggest that training in EBP is attractive to patient representatives with outputs perceived important. Organizers should consider the principles of EBP when planning such training.

Key Concepts for Informed Health Choices: a framework for helping people learn how to assess treatment claims and make informed choices.

Many claims about the effects of treatments, though well intentioned, are wrong. Indeed, they are sometimes deliberately misleading to serve interests other than the well-being of patients and the public. People need to know how to spot unreliable treatment claims so that they can protect themselves and others from harm. The ability to assess the trustworthiness of treatment claims is often lacking. Acquiring this ability depends on being familiar with, and correctly applying, some key concepts, for example, that' association is not the same as causation.' The Informed Health Choices (IHC) Project has identified 36 such concepts and shown that people can be taught to use them in decision making. A randomised trial in Uganda, for example, showed that primary school children with poor reading skills could be taught to apply 12 of the IHC Key Concepts. The list of IHC Key Concepts has proven to be effective in providing a framework for developing and evaluating IHC resources to help children to think critically about treatment claims. The list also provides a framework for retrieving, coding and organising other teaching and learning materials for learners of any age. It should help teachers, researchers, clinicians, and patients to structure critical thinking about the trustworthiness of claims about treatment effects.

Key Concepts for assessing claims about treatment effects and making well-informed treatment choices.

Background: The Informed Health Choices (IHC) Key Concepts are standards for judgement, or principles for evaluating the trustworthiness of treatment claims and treatment comparisons (evidence) used to support claims, and for making treatment choices. The list of concepts provides a framework, or starting point, for teachers, journalists and other intermediaries for identifying and developing resources (such as longer explanations, examples, games and interactive applications) to help people to understand and apply the concepts. The first version of the list was published in 2015 and has been updated yearly since then. We report here the changes that have been made from when the list was first published up to the current (2018) version. Methods: We developed the IHC Key Concepts by searching the literature and checklists written for the public, journalists, and health professionals; and by considering concepts related to assessing the certainty of evidence about the effects of treatments. We have revised the Key Concepts yearly, based on feedback and suggestions; and learning from using the IHC Key Concepts, other relevant frameworks, and adaptation of the IHC Key Concepts to other types of interventions besides treatments. Results: We have made many changes since the Key Concepts were first published in 2015. There are now 44 Key Concepts compared to the original 32; the concepts have been reorganised from six to three groups; we have added higher-level concepts in each of those groups; we have added short titles; and we have made changes to many of the concepts. Conclusions: The IHC Key Concepts have proven useful in designing learning resources, evaluating them, and organising them. We will continue to revise the IHC Key Concepts in response to feedback. We welcome suggestions for how to do this.

Establishing a library of resources to help people understand key concepts in assessing treatment claims-The "Critical thinking and Appraisal Resource Library" (CARL).

BACKGROUND: People are frequently confronted with untrustworthy claims about the effects of treatments. Uncritical acceptance of these claims can lead to poor, and sometimes dangerous, treatment decisions, and wasted time and money. Resources to help people learn to think critically about treatment claims are scarce, and they are widely scattered. Furthermore, very few learning-resources have been assessed to see if they improve knowledge and behavior. OBJECTIVES: Our objectives were to develop the Critical thinking and Appraisal Resource Library (CARL). This library was to be in the form of a database containing learning resources for those who are responsible for encouraging critical thinking about treatment claims, and was to be made available online. We wished to include resources for groups we identified as 'intermediaries' of knowledge, i.e. teachers of schoolchildren, undergraduates and graduates, for example those teaching evidence-based medicine, or those communicating treatment claims to the public. In selecting resources, we wished to draw particular attention to those resources that had been formally evaluated, for example, by the creators of the resource or independent research groups. METHODS: CARL was populated with learning-resources identified from a variety of sources-two previously developed but unmaintained inventories; systematic reviews of learning-interventions; online and database searches; and recommendations by members of the project group and its advisors. The learning-resources in CARL were organised by 'Key Concepts' needed to judge the trustworthiness of treatment claims, and were made available online by the James Lind Initiative in Testing Treatments interactive (TTi) English (www.testingtreatments.org/category/learning-resources).TTi English also incorporated the database of Key Concepts and the Claim Evaluation Tools developed through the Informed Healthcare Choices (IHC) project (informedhealthchoices.org). RESULTS: We have created a database of resources called CARL, which currently contains over 500 open-access learning-resources in a variety of formats: text, audio, video, webpages, cartoons, and lesson materials. These are aimed primarily at 'Intermediaries', that is, 'teachers', 'communicators', 'advisors', 'researchers', as well as for independent 'learners'. The resources included in CARL are currently accessible at www.testingtreatments.org/category/learning-resources. CONCLUSIONS: We hope that ready access to CARL will help to promote the critical thinking about treatment claims, needed to help improve healthcare choices.

Does the use of the Informed Healthcare Choices (IHC) primary school resources improve the ability of grade-5 children in Uganda to assess the trustworthiness of claims about the effects of treatments: protocol for a cluster-randomised trial.

BACKGROUND: The ability to appraise claims about the benefits and harms of treatments is crucial for informed health care decision-making. This research aims to enable children in East African primary schools (the clusters) to acquire and retain skills that can help them make informed health care choices by improving their ability to obtain, process and understand health information. The trial will evaluate (at the individual participant level) whether specially designed learning resources can teach children some of the key concepts relevant to appraising claims about the benefits and harms of health care interventions (treatments). METHODS: This is a two-arm, cluster-randomised trial with stratified random allocation. We will recruit 120 primary schools (the clusters) between April and May 2016 in the central region of Uganda. We will stratify participating schools by geographical setting (rural, semi-urban, or urban) and ownership (public or private). The Informed Healthcare Choices (IHC) primary school resources consist of a textbook and a teachers' guide. Each of the students in the intervention arm will receive a textbook and attend nine lessons delivered by their teachers during a school term, with each lesson lasting 80 min. The lessons cover 12 key concepts that are relevant to assessing claims about treatments and making informed health care choices. The second arm will carry on with the current primary school curriculum. We have designed the Claim Evaluation Tools to measure people's ability to apply key concepts related to assessing claims about the effects of treatments and making informed health care choices. The Claim Evaluation Tools use multiple choice questions addressing each of the 12 concepts covered by the IHC school resources. Using the Claim Evaluation Tools we will measure two primary outcomes: (1) the proportion of children who 'pass', based on an absolute standard and (2) their average scores. DISCUSSION: As far as we are aware this is the first randomised trial to assess whether key concepts needed to judge claims about the effects of treatment can be taught to primary school children. Whatever the results, they will be relevant to learning how to promote critical thinking about treatment claims. Trial status: the recruitment of study participants was ongoing at the time of manuscript submission. TRIAL REGISTRATION: Pan African Clinical Trial Registry, trial identifier: PACTR201606001679337 . Registered on 13 June 2016.

Measuring ability to assess claims about treatment effects: a latent trait analysis of items from the 'Claim Evaluation Tools' database using Rasch modelling.

BACKGROUND: The Claim Evaluation Tools database contains multiple-choice items for measuring people's ability to apply the key concepts they need to know to be able to assess treatment claims. We assessed items from the database using Rasch analysis to develop an outcome measure to be used in two randomised trials in Uganda. Rasch analysis is a form of psychometric testing relying on Item Response Theory. It is a dynamic way of developing outcome measures that are valid and reliable. OBJECTIVES: To assess the validity, reliability and responsiveness of 88 items addressing 22 key concepts using Rasch analysis. PARTICIPANTS: We administrated four sets of multiple-choice items in English to 1114 people in Uganda and Norway, of which 685 were children and 429 were adults (including 171 health professionals). We scored all items dichotomously. We explored summary and individual fit statistics using the RUMM2030 analysis package. We used SPSS to perform distractor analysis. RESULTS: Most items conformed well to the Rasch model, but some items needed revision. Overall, the four item sets had satisfactory reliability. We did not identify significant response dependence between any pairs of items and, overall, the magnitude of multidimensionality in the data was acceptable. The items had a high level of difficulty. CONCLUSION: Most of the items conformed well to the Rasch model's expectations. Following revision of some items, we concluded that most of the items were suitable for use in an outcome measure for evaluating the ability of children or adults to assess treatment claims.

Measuring ability to assess claims about treatment effects: the development of the 'Claim Evaluation Tools'.

OBJECTIVES: To describe the development of the Claim Evaluation Tools, a set of flexible items to measure people's ability to assess claims about treatment effects. SETTING: Methodologists and members of the community (including children) in Uganda, Rwanda, Kenya, Norway, the UK and Australia. PARTICIPANTS: In the iterative development of the items, we used purposeful sampling of people with training in research methodology, such as teachers of evidence-based medicine, as well as patients and members of the public from low-income and high-income countries. Development consisted of 4 processes: (1) determining the scope of the Claim Evaluation Tools and development of items; (2) expert item review and feedback (n=63); (3) cognitive interviews with children and adult end-users (n=109); and (4) piloting and administrative tests (n=956). RESULTS: The Claim Evaluation Tools database currently includes a battery of multiple-choice items. Each item begins with a scenario which is intended to be relevant across contexts, and which can be used for children (from age 10  and above), adult members of the public and health professionals. People with expertise in research methods judged the items to have face validity, and end-users judged them relevant and acceptable in their settings. In response to feedback from methodologists and end-users, we simplified some text, explained terms where needed, and redesigned formats and instructions. CONCLUSIONS: The Claim Evaluation Tools database is a flexible resource from which researchers, teachers and others can design measurement instruments to meet their own requirements. These evaluation tools are being managed and made freely available for non-commercial use (on request) through Testing Treatments interactive (testingtreatments.org). TRIAL REGISTRATION NUMBERS: PACTR201606001679337 and PACTR201606001676150; Pre-results.

Effects of the Informed Health Choices primary school intervention on the ability of children in Uganda to assess the reliability of claims about treatment effects: a cluster-randomised controlled trial.

BACKGROUND: Claims about what improves or harms our health are ubiquitous. People need to be able to assess the reliability of these claims. We aimed to evaluate an intervention designed to teach primary school children to assess claims about the effects of treatments (ie, any action intended to maintain or improve health). METHODS: In this cluster-randomised controlled trial, we included primary schools in the central region of Uganda that taught year-5 children (aged 10-12 years). We excluded international schools, special needs schools for children with auditory and visual impairments, schools that had participated in user-testing and piloting of the resources, infant and nursery schools, adult education schools, and schools that were difficult for us to access in terms of travel time. We randomly allocated a representative sample of eligible schools to either an intervention or control group. Intervention schools received the Informed Health Choices primary school resources (textbooks, exercise books, and a teachers' guide). Teachers attended a 2 day introductory workshop and gave nine 80 min lessons during one school term. The lessons addressed 12 concepts essential to assessing claims about treatment effects and making informed health choices. We did not intervene in the control schools. The primary outcome, measured at the end of the school term, was the mean score on a test with two multiple-choice questions for each of the 12 concepts and the proportion of children with passing scores on the same test. This trial is registered with the Pan African Clinical Trial Registry, number PACTR201606001679337. FINDINGS: Between April 11, 2016, and June 8, 2016, 2960 schools were assessed for eligibility; 2029 were eligible, and a random sample of 170 were invited to recruitment meetings. After recruitment meetings, 120 eligible schools consented and were randomly assigned to either the intervention group (n=60, 76 teachers and 6383 children) or control group (n=60, 67 teachers and 4430 children). The mean score in the multiple-choice test for the intervention schools was 62·4% (SD 18·8) compared with 43·1% (15·2) for the control schools (adjusted mean difference 20·0%, 95% CI 17·3-22·7; p<0·00001). In the intervention schools, 3967 (69%) of 5753 children achieved a predetermined passing score (≥13 of 24 correct answers) compared with 1186 (27%) of 4430 children in the control schools (adjusted difference 50%, 95% CI 44-55). The intervention was effective for children with different levels of reading skills, but was more effective for children with better reading skills. INTERPRETATION: The use of the Informed Health Choices primary school learning resources, after an introductory workshop for the teachers, led to a large improvement in the ability of children to assess claims about the effects of treatments. The results show that it is possible to teach primary school children to think critically in schools with large student to teacher ratios and few resources. Future studies should address how to scale up use of the resources, long-term effects, including effects on actual health choices, transferability to other countries, and how to build on this programme with additional primary and secondary school learning resources. FUNDING: Research Council of Norway.

Effects of the Informed Health Choices podcast on the ability of parents of primary school children in Uganda to assess claims about treatment effects: a randomised controlled trial.

BACKGROUND: As part of the Informed Health Choices project, we developed a podcast called The Health Choices Programme to help improve the ability of people to assess claims about the benefits and harms of treatments. We aimed to evaluate the effects of the podcast on the ability of parents of primary school children in Uganda to assess claims about the effects of treatments. METHODS: We did this randomised controlled trial in central Uganda. We recruited parents of children aged 10-12 years who were in their fifth year of school at 35 schools that were participating in a linked trial of the Informed Health Choices primary school resources. The parents were randomly allocated (1:1), via a web-based random number generator with block sizes of four and six, to listen to either the Informed Health Choices podcast (intervention group) or typical public service announcements about health issues (control group). Randomisation was stratified by parents' highest level of formal education attained (primary school, secondary school, or tertiary education) and the allocation of their children's school in the trial of the primary school resources (intervention vs control). The primary outcome, measured after listening to the entire podcast, was the mean score and the proportion of parents with passing scores on a test with two multiple choice questions for each of nine key concepts essential to assessing claims about treatments (18 questions in total). We did intention-to-treat analyses. This trial is registered with the Pan African Clinical Trial Registry, number PACTR201606001676150. FINDINGS: We recruited parents between July 21, 2016, and Oct 7, 2016. We randomly assigned 675 parents to the podcast group (n=334) or the public service announcement group (n=341); 561 (83%) participants completed follow-up. The mean score for parents in the podcast group was 67·8% (SD 19·6) compared with 52·4% (17·6) in the control group (adjusted mean difference 15·5%, 95% CI 12·5-18·6; p<0·0001). In the podcast group, 203 (71%) of 288 parents had a predetermined passing score (≥11 of 18 correct answers) compared with 103 (38%) of 273 parents in the control group (adjusted difference 34%, 95% CI 26-41; p<0·0001). No adverse events were reported. INTERPRETATION: Listening to the Informed Health Choices podcast led to a large improvement in the ability of parents to assess claims about the effects of treatments. Future studies should assess the long-term effects of use of the podcast, the effects on actual health choices and outcomes, and how transferable our findings are to other countries. FUNDING: Research Council of Norway.

[Patient volume and quality in surgery for abdominal aortic aneurysm]. / Pasientvolum og kvalitet ved kirurgi for abdominale aortaaneurismer.

BACKGROUND Patient volume is assumed to affect quality, whereby complex procedures are best performed by those who perform them frequently. We have conducted a systematic review of the research on the association between patient volume and quality of vascular surgery. In this article we describe the outcomes for abdominal aortic aneurysm surgery.MATERIAL AND METHOD We undertook systematic searches in relevant databases. We searched for systematic reviews, and randomised and observational studies. The search was concluded in December 2015. We have summarised the results descriptively and assessed the overall quality of the evidence.RESULTS Forty-six observational studies fulfilled our inclusion criteria. We found a possible association for both hospital and surgeon volume. Higher patient volume may possibly be associated with lower 30-day mortality and lower hospital mortality for both open and endovascular surgery. Although the association appears to apply to both elective and acute hospitalisations, there is greater uncertainty with regard to the most ill patients. For hospital volume there may also be fewer complications for open and endovascular surgery, as well as for all surgery assessed as a whole. We considered the evidence base to be medium to very low quality.INTERPRETATION We found a possible correlation between patient volume and quality indicators such as mortality and complications. It may be advantageous to allocate planned procedures to institutions and surgeons with high volume, while this is less certain with regard to acute hospitalisations.

Can an educational podcast improve the ability of parents of primary school children to assess the reliability of claims made about the benefits and harms of treatments: study protocol for a randomised controlled trial.

BACKGROUND: Claims made about the effects of treatments are very common in the media and in the population more generally. The ability of individuals to understand and assess such claims can affect their decisions and health outcomes. Many people in both low- and high-income countries have inadequate aptitude to assess information about the effects of treatments. As part of the Informed Healthcare Choices project, we have prepared a series of podcast episodes to help improve people's ability to assess claims made about treatment effects. We will evaluate the effect of the Informed Healthcare Choices podcast on people's ability to assess claims made about the benefits and harms of treatments. Our study population will be parents of primary school children in schools with limited educational and financial resources in Uganda. METHODS: This will be a two-arm, parallel-group, individual-randomised trial. We will randomly allocate consenting participants who meet the inclusion criteria for the trial to either listen to nine episodes of the Informed Healthcare Choices podcast (intervention) or to listen to nine typical public service announcements about health issues (control). Each podcast includes a story about a treatment claim, a message about one key concept that we believe is important for people to be able to understand to assess treatment claims, an explanation of how that concept applies to the claim, and a second example illustrating the concept. We designed the Claim Evaluation Tools to measure people's ability to apply key concepts related to assessing claims made about the effects of treatments and making informed health care choices. The Claim Evaluation Tools that we will use include multiple-choice questions addressing each of the nine concepts covered by the podcast. Using the Claim Evaluation Tools, we will measure two primary outcomes: (1) the proportion that 'pass', based on an absolute standard and (2) the average score. DISCUSSION: As far as we are aware this is the first randomised trial to assess the use of mass media to promote understanding of the key concepts needed to judge claims made about the effects of treatments. TRIAL REGISTRATION: Pan African Clinical Trials Registry, PACTR201606001676150. Registered on 12 June 2016. http://www.pactr.org/ATMWeb/appmanager/atm/atmregistry?dar=true&tNo=PACTR201606001676150 .