Conducting two evidence syntheses in six weeks - experiences with and evaluation of a pilot project.

BACKGROUND: Evidence synthesis organisations are trying to meet commissioners' needs for rapid responses to their evidence synthesis commissions. In this project we piloted an intensive process, working to complete evidence syntheses within six-weeks, rather than the standard lead time of 4-6 months. Our objective was to explore how researchers experience working intensively, identify barriers and facilitators, and determine how a more intensive approach to evidence synthesis could be more systematically introduced in the future. METHODS: In a pre-planning phase, an intensive work group was established, and two commissions were selected for this pilot project. The evidence synthesis process was divided into two phases: planning and intensive. The planning phase, involved scheduling the intensive phase, exploring new digital tools, and identifying peer reviewers. The intensive phase encompassed the entire evidence synthesis process. Two review teams were formed, each with a team lead supported by a process lead and leadership contact point. Throughout the project, teams engaged in reflective meetings to evaluate and adjust processes as needed. RESULTS: During the planning phase, teams identified significant uncertainties regarding scopes, research questions, and inclusion criteria. To address this, they engaged with commissioners earlier than originally planned, clarified these aspects, and prepared protocols. Despite some minor deviations from the original plan, both reviews were completed on schedule, with one team expanding their scope due to the absence of eligible studies. Teams operated flexibly, held regular meetings, and found the process seamless due to fewer interruptions. Machine learning tools facilitated rapid study selection. The process lead role, created to guide and evaluate the project, proved beneficial, providing structure and support, although clearer role delineation with the leadership contact point could have improved efficiency. CONCLUSIONS: Overall, the intensive process fostered focus and productivity, allowing teams to manage short-term deliverables effectively. The researchers preferred working intensively with one evidence synthesis over being involved with many projects at the same time. They felt that time use was more effective, and they were able to complete the tasks in a focused way. However, there are several implications that should be considered before implementing an intensive approach in future evidence syntheses.

Tools for assessing the methodological limitations of a QES-a short note.

The increasing prevalence and application of qualitative evidence syntheses (QES) in decision-making processes underscore the need for robust tools to assess the methodological limitations of a completed QES. This commentary discusses the limitations of three existing tools and presents the authors' efforts to address this gap. Through a simple comparative analysis, the three tools are examined in terms of their coverage of essential topic areas. The examination finds that existing assessment tools lack comprehensive coverage, clarity, and grounding in qualitative research principles. The authors advocate for the development of a new collaboratively developed evidence-based tool rooted in qualitative methodology and best practice methods. The conclusion emphasizes the necessity of a tool that can provide a comprehensive judgement on the methodological limitations of a QES, addressing the needs of end-users, and ultimately enhancing the trustworthiness of QES findings in decision-making processes.

Machine learning in systematic reviews: Comparing automated text clustering with Lingo3G and human researcher categorization in a rapid review.

Systematic reviews are resource-intensive. The machine learning tools being developed mostly focus on the study identification process, but tools to assist in analysis and categorization are also needed. One possibility is to use unsupervised automatic text clustering, in which each study is automatically assigned to one or more meaningful clusters. Our main aim was to assess the usefulness of an automated clustering method, Lingo3G, in categorizing studies in a simplified rapid review, then compare performance (precision and recall) of this method compared to manual categorization. We randomly assigned all 128 studies in a review to be coded by a human researcher blinded to cluster assignment (mimicking two independent researchers) or by a human researcher non-blinded to cluster assignment (mimicking one researcher checking another's work). We compared time use, precision and recall of manual categorization versus automated clustering. Automated clustering and manual categorization organized studies by population and intervention/context. Automated clustering failed to identify two manually identified categories but identified one additional category not identified by the human researcher. We estimate that automated clustering has similar precision to both blinded and non-blinded researchers (e.g., 88% vs. 89%), but higher recall (e.g., 89% vs. 84%). Manual categorization required 49% more time than automated clustering. Using a specific clustering algorithm, automated clustering can be helpful with categorization of and identifying patterns across studies in simpler systematic reviews. We found that the clustering was sensitive enough to group studies according to linguistic differences that often corresponded to the manual categories.

Community and Drug Distributor Perceptions and Experiences of Mass Drug Administration for the Elimination of Lymphatic Filariasis: A Rapid Review of Qualitative Research.

STUDY OBJECTIVES: This article presents findings from a rapid review of qualitative research conducted to inform decision makers about community and drug distributor perceptions and experiences of mass drug administration campaigns for the elimination of lymphatic filariasis. We focused on questions related to acceptability of the mass drug administration campaigns within these groups and their thoughts around the feasibility of planning and carrying out the campaigns. METHODS: We carried out a systematic search in five databases to identify potential studies. We included studies that focused on community members and drug distributors and used qualitative methods for data collection and analysis. We conducted a thematic framework analysis using the Supporting the Use of Research Evidence framework. Due to time constraints, one author conducted the screening, extraction and data analysis. FINDINGS: Studies found that communities lack knowledge and information about lymphatic filariasis and the mass drug administration campaigns and that this can have an impact on how many community members choose to take medication. Health workers often had a good understanding that lymphatic filariasis was a problem in their setting, of its cause and mode of transmission and that hydrocele and elephantiasis had the same cause. However, this knowledge was not as prevalent amongst community drug distributors who often had misconceptions surrounding the topic. Furthermore, studies found that the length, timing, level of community and health system involvement, access to care for side effects, inadequate numbers of drug distributors and supervisors and motivation of drug distributors influenced participation in mass drug administration campaigns. Finally, the inadequate training of drug distributors could influence community trust in the mass drug administration program and the drug distributor themselves if there was a perception that the person was uninformed or not trained to carry out their tasks.