Needs and feasibility of living systematic reviews (LSRs): Experience from LSRs on COVID-19 vaccine effectiveness.

During 2021 and 2023, a team of researchers at the Robert Koch Institute (RKI) and partnering institutions conducted two living systematic reviews (LSRs) on the effectiveness of COVID-19 vaccines in different age groups to inform recommendations of the Standing Committee on Vaccination in Germany (Ständige Impfkommission, STIKO). Based on our experience from the realization of these LSRs, we developed certain criteria to assess the needs and feasibility of conducting LSRs. Combining these with previously established criteria, we developed the following set to inform future planning of LSRs for STIKO: Needs criterion (N)1: Relevance of the research question, N2: Certainty of evidence (CoE) at baseline; N3: Expected need for Population-Intervention-Comparator-Outcome (PICO) adaptations; N4: Expected new evidence over time; N5: Expected impact of new evidence on CoE; Feasibility criterion (F)1: Availability of sufficient human resources; F2: Feasibility of timely dissemination of the results to inform decision-making. For each criterion we suggest rating options which may support the decision to conduct an LSR or other forms of evidence synthesis when following the provided flowchart. The suggested criteria were developed on the basis of the experiences from exemplary reviews in a specific research field (i.e., COVID-19 vaccination), and did not follow a formal development or validation process. However, these criteria might also be useful to assess whether questions from other research fields can and should be answered using the LSR approach, or assist in determining whether the use of an LSR is sensible and feasible for specific questions in health policy and practice.

[Development of criteria for the prospective assessment of the need for updating guideline recommendations: The AGIL criteria]. / Entwicklung von Kriterien für die prospektive Einschätzung des Aktualisierungsbedarfs von Leitlinienempfehlungen: AGIL-Kriterien.

BACKGROUND: Evidence-based guideline and vaccination recommendations should continuously be updated to appropriately support health care decisions. However, resources for updating guidelines are often limited. The aim of this project was to develop a list of criteria for the prospective assessment of the need for updating individual guideline or vaccination recommendations, which can be applied from the time a guideline or guideline update is finalised. METHODS: In this article we describe the development of the AGIL criteria (Assessment of Guidelines for Updating Recommendations). The AGIL criteria were developed by experienced scientists and experts in the field of guideline development in a multi-step process. The five steps included: 1) development of an initial list of criteria by the project team; 2) online survey of guideline experts on the initial version of the criteria list; 3) revision of the criteria list based on the results of the online survey; 4) workshop on the criteria list at the EbM Congress 2023; 5) creation of version 1.0 of the AGIL criteria based on the workshop results. RESULTS: The initial list included the following three criteria: 1) relevance of the question 2) availability of new relevant evidence, and 3) impact of potentially new evidence. The response rate of the online survey for fully completed questionnaires was 31.0% (N=195; 630 guideline experts were contacted by email). For 90.3% (n=176) of the respondents, the criteria list included all essential aspects for assessing the need for updating guideline recommendations. More than three quarters of respondents rated the importance of the three criteria as "very important" or "important" (criteria 1-3: 75.3%, 86.1%, 85.2%) and - with the exception of criterion 1 - comprehensibility as "very comprehensible" or "comprehensible" (criteria 1-3: 58.4%, 75.9%, 78.5%). The results of the online survey and the workshop generally confirmed the three criteria with their two sub-questions. The incorporation of all feedback resulted in the AGIL criteria (version 1.0), recapping: 1) relevance of the question regarding a) PICO components and b) other factors, e.g. epidemiological aspects; 2) availability of new evidence a) on health-related benefits and harms and b) on other decision factors, e.g. feasibility, acceptability; 3) impact of new evidence a) on the certainty of evidence on which the recommendation is based and b) on the present recommendation, e.g. DISCUSSION: The moderate response rate of the online survey may have limited its representativeness. Nevertheless, we consider the response rate to be satisfactory in this research context. The inclusion of many experts in the online survey and the EbM Congress workshop is a strength of the project and supports the quality of the results. CONCLUSIONS: The AGIL criteria provide a structured guidance for the prospective assessment of the need for updating individual guideline recommendations and other evidence-based recommendations. The implementation and evaluation of the AGIL criteria 1.0 in a field test is planned.

Safety and effectiveness of vaccines against COVID-19 in children aged 5-11 years: a systematic review and meta-analysis.

BACKGROUND: To date, more than 761 million confirmed SARS-CoV-2 infections have been recorded globally, and more than half of all children are estimated to be seropositive. Despite high SARS-CoV-2 infection incidences, the rate of severe COVID-19 in children is low. We aimed to assess the safety and efficacy or effectiveness of COVID-19 vaccines approved in the EU for children aged 5-11 years. METHODS: In this systematic review and meta-analysis, we included studies of any design identified through searching the COVID-19 L·OVE (living overview of evidence) platform up to Jan 23, 2023. We included studies with participants aged 5-11 years, with any COVID-19 vaccine approved by the European Medicines Agency-ie, mRNA vaccines BNT162b2 (Pfizer-BioNTech), BNT162b2 Bivalent (against original strain and omicron [BA.4 or BA.5]), mRNA-1273 (Moderna), or mRNA-1273.214 (against original strain and omicron BA.1). Efficacy and effectiveness outcomes were SARS-CoV-2 infection (PCR-confirmed or antigen-test confirmed), symptomatic COVID-19, hospital admission due to COVID-19, COVID-19-related mortality, multisystem inflammatory syndrome in children (MIS-C), and long-term effects of COVID-19 (long COVID or post-COVID-19 condition as defined by study investigators or per WHO definition). Safety outcomes of interest were serious adverse events, adverse events of special interest (eg, myocarditis), solicited local and systemic events, and unsolicited adverse events. We assessed risk of bias and rated the certainty of evidence (CoE) using the Grading of Recommendations Assessment, Development and Evaluation approach. This study was prospectively registered with PROSPERO, CRD42022306822. FINDINGS: Of 5272 screened records, we included 51 (1·0%) studies (n=17 [33%] in quantitative synthesis). Vaccine effectiveness after two doses against omicron infections was 41·6% (95% CI 28·1-52·6; eight non-randomised studies of interventions [NRSIs]; CoE low), 36·2% (21·5-48·2; six NRSIs; CoE low) against symptomatic COVID-19, 75·3% (68·0-81·0; six NRSIs; CoE moderate) against COVID-19-related hospitalisations, and 78% (48-90, one NRSI; CoE very low) against MIS-C. Vaccine effectiveness against COVID-19-related mortality was not estimable. Crude event rates for deaths in unvaccinated children were less than one case per 100 000 children, and no events were reported for vaccinated children (four NRSIs; CoE low). No study on vaccine effectiveness against long-term effects was identified. Vaccine effectiveness after three doses was 55% (50-60; one NRSI; CoE moderate) against omicron infections, and 61% (55-67; one NRSI; CoE moderate) against symptomatic COVID-19. No study reported vaccine efficacy or effectiveness against hospitalisation following a third dose. Safety data suggested no increased risk of serious adverse events (risk ratio [RR] 0·83 [95% CI 0·21-3·33]; two randomised controlled trials; CoE low), with approximately 0·23-1·2 events per 100 000 administered vaccines reported in real-life observations. Evidence on the risk of myocarditis was uncertain (RR 4·6 [0·1-156·1]; one NRSI; CoE low), with 0·13-1·04 observed events per 100 000 administered vaccines. The risk of solicited local reactions was 2·07 (1·80-2·39; two RCTs; CoE moderate) after one dose and 2·06 (1·70-2·49; two RCTs; CoE moderate) after two doses. The risk of solicited systemic reactions was 1·09 (1·04-1·16; two RCTs; CoE moderate) after one dose and 1·49 (1·34-1·65; two RCTs; CoE moderate) after two doses. The risk of unsolicited adverse events after two doses (RR 1·21 [1·07-1·38]; CoE moderate) was higher among mRNA-vaccinated compared with unvaccinated children. INTERPRETATION: In children aged 5-11 years, mRNA vaccines are moderately effective against infections with the omicron variant, but probably protect well against COVID-19 hospitalisations. Vaccines were reactogenic but probably safe. Findings of this systematic review can serve as a basis for public health policy and individual decision making on COVID-19 vaccination in children aged 5-11 years. FUNDING: German Federal Joint Committee.