Expert opinion on a consensus-based checklist for the critical appraisal of cost-of-illness (COI) studies: qualitative interviews.

OBJECTIVES: This study explored experts' views on the development of a proposed checklist for cost-of-illness (COI) studies. It also investigated experts' perspectives on the use of COI studies and quality/critical appraisal tools used for COI studies as well as their experiences with the use of these tools. METHODS: Semi-structured, open-ended interviews were conducted with health economists and other experts working with COI studies and with experience of developing health economic guidelines or checklists. Participants were selected purposively using network and snowball sampling. A framework approach was applied for the thematic data analysis. Findings were reported narratively. RESULTS: Twenty-one experts from eleven different countries were interviewed. COI studies were found to be relevant to estimate the overall burden of a disease, to draw attention to disease areas, to understand different cost components, to explain cost variability, to inform decision making, and to provide input for full economic evaluations. Experts reported a lack of a standardized critical appraisal tool for COI studies. Their experience related predominantly to guidelines and checklists designed for full economic evaluations to review and assess COI studies. The following themes emerged when discussing the checklist: (i) the need for a critical appraisal tool, (ii) format and practicality, (iii) assessing the questions, (iv) addressing subjectivity, and (v) guidance requirements. CONCLUSIONS: The interviews provided relevant input for the development of a checklist for COI studies that could be used as a minimum standard and for international application. The interviews confirmed the important need for a checklist for the critical appraisal of COI studies.

CAT HPPR: a critical appraisal tool to assess the quality of systematic, rapid, and scoping reviews investigating interventions in health promotion and prevention.

BACKGROUND: For over three decades researchers have developed critical appraisal tools (CATs) for assessing the scientific quality of research overviews. Most established CATs for reviews in evidence-based medicine and evidence-based public health (EBPH) focus on systematic reviews (SRs) with studies on experimental interventions or exposure included. EBPH- and implementation-oriented organisations and decision-makers, however, often seek access to rapid reviews (RRs) or scoping reviews (ScRs) for rapid evidence synthesis and research field exploration. Until now, no CAT is available to assess the quality of SRs, RRs, and ScRs following a unified approach. We set out to develop such a CAT. METHODS: The development process of the Critical Appraisal Tool for Health Promotion and Prevention Reviews (CAT HPPR) included six phases: (i) the definition of important review formats and complementary approaches, (ii) the identification of relevant CATs, (iii) prioritisation, selection and adaptation of quality criteria using a consensus approach, (iv) development of the rating system and bilingual guidance documents, (v) engaging with experts in the field for piloting/optimising the CAT, and (vi) approval of the final CAT. We used a pragmatic search approach to identify reporting guidelines/standards (n = 3; e.g. PRISMA, MECIR) as well as guidance documents (n = 17; e.g. for reviews with mixed-methods approach) to develop working definitions for SRs, RRs, ScRs, and other review types (esp. those defined by statistical methods or included data sources). RESULTS: We successfully identified 14 relevant CATs, predominantly for SRs (e.g. AMSTAR 2), and extracted 46 items. Following consensual discussions 15 individual criteria were included in our CAT and tailored to the review types of interest. The CAT was piloted with 14 different reviews which were eligible to be included in a new German database looking at interventions in health promotion and prevention in different implementation settings. CONCLUSIONS: The newly developed CAT HPPR follows a unique uniformed approach to assess a set of heterogeneous reviews (e.g. reviews from problem identification to policy evaluations) to assist end-users needs. Feedback of external experts showed general feasibility and satisfaction with the tool. Future studies should further formally test the validity of CAT HPPR using larger sets of reviews.

The Critical Appraisal Tool for Anatomical Meta-analysis: A framework for critically appraising anatomical meta-analyses.

The hallmark of evidence-based medicine is the meta-analysis (MA). For much of its rich history, the field of anatomy has been dominated by descriptive, cadaveric studies. In the last two decades, quantitative measurements and statistical analyses have frequently accompanied such studies. These studies have directly led to the publication of anatomical MAs, which have ushered in the exciting field of evidence-based anatomy. Although critical appraisal tools exist for clinical MAs, none of them are specifically tailored for anatomical MAs. Therefore, the purpose of this article is to provide a framework by which clinical anatomists and others can critically appraise anatomical MAs using the Critical Appraisal Tool for Anatomical Meta-analysis (CATAM). Using a running example from a recently published MA, we show how to use the CATAM rubric in a step-by-step fashion. Each scored section of the CATAM rubric is summated into a total score (maximum 50 points). This score is then referenced to a conversion chart, which assigns a qualitative value to the MA in a range from "very good" to "poor." Future studies can investigate the interrater reliability of the instrument, and possibly subject the CATAM rubric to a Delphi panel. As anatomical MAs become more commonplace at surgical grand rounds and journal clubs in academic medical centers throughout the world, we hope that the CATAM rubric can help facilitate meaningful discussions about the quality and clinical relevance of anatomical MAs.

Factors that enhance compliance with the Surgical Safety Checklist.

BACKGROUND: The World Health Organization (WHO) has reported that its Surgical Safety Checklist (SSC) has resulted in significant reductions in morbidity and mortality. Despite its proven success, meaningful compliance with the Surgical Safety Checklist initiative has been low. AIMS: The authors sought to identify and explore published research on factors that enhance compliance with the SSC within surgical team members. METHODS: A review of the literature published between January 2017 and January 2021 was undertaken. Six databases were searched, and 1340 studies were screened for eligibility. The 17 studies included were critically appraised using the Crowe Critical Appraisal Tool. FINDINGS: Three main themes were identified: training and innovations; process adaptations and team leadership. CONCLUSION: This review of the literature draws attention to the complexities of checklist compliance and identifies the need for training, leadership and adaptation to new safety processes.

Critical appraisal tools used in systematic reviews of in vitro cell culture studies: A methodological study.

Systematic reviews (SRs) of preclinical studies are marked with poor methodological quality. In vitro studies lack assessment tools to improve the quality of preclinical research. This methodological study aimed to identify, collect, and analyze SRs based on cell culture studies to highlight the current appraisal tools utilized to support the development of a validated critical appraisal tool for cell culture in vitro research. SRs, scoping reviews, and meta-analyses that included cell culture studies and used any type of critical appraisal tool were included. Electronic search, study selection, data collection and methodological quality (MQ) assessment tool were realized. Further, statistical analyses regarding possible associations and correlations between MQ and collected data were performed. After the screening process, 82 studies remained for subsequent analysis. A total of 32 different appraisal tools were identified. Approximately 60% of studies adopted pre-structured tools not designed for cell culture studies. The most frequent instruments were SYRCLE (n = 14), OHAT (n = 9), Cochrane Collaboration's tool (n = 7), GRADE (n = 6), CONSORT (n = 5), and ToxRTool (n = 5). The studies were divided into subgroups to perform statistical analyses. A significant association (OR = 5.00, 95% CI = 1.54-16.20, p = 0.008) was found between low MQ and chronic degenerative disorders as topic of SR. Several challenges in collecting information from the included studies led to some modifications related to the previously registered protocol. These results may serve as a basis for further development of a critical appraisal tool for cell culture studies capable of capturing all the essential factors related to preclinical research, therefore enhancing the practice of evidence-based.

AMSTAR 2 is only partially applicable to systematic reviews of non-intervention studies: a meta-research study.

OBJECTIVES: A measurement tool to assess systematic reviews 2 (AMSTAR 2) was originally developed for systematic reviews (SRs) of health-care interventions. The aim of this study was to assess the applicability of AMSTAR 2 to SRs of non-intervention studies. STUDY DESIGN AND SETTING: This was a meta-research study. We used 20 SRs for each of the following four types of SRs: Diagnostic Test Accuracy reviews, Etiology and/or Risk reviews, Prevalence and/or Incidence reviews, and Prognostic reviews (80 in total). Three authors applied AMSTAR 2 independently to each included SRs. Then, the authors assessed the applicability of each item to that SR type and any SR type. RESULTS: Researchers unanimously indicated that 7 of 16 AMSTAR 2 items were applicable for all four specific SR types and any SR type (items 2, 5, 6, 7, 10, 14 and 16), but 8 of 16 items for any SR type. These items could cover generic SR methods that do not depend on a specific SR type. CONCLUSION: AMSTAR 2 is only partially applicable for non-intervention SRs. There is a need to adapt/extend AMSTAR 2 for SRs of non-intervention studies. Our study can help to further define generic methodological aspects shared across SR types and methodological expectations for non-intervention SRs.

The use of a modified Delphi technique to develop a critical appraisal tool for clinical pharmacokinetic studies.

BACKGROUND: Critical appraisal aids in assessing the quality of scientific literature, which is central to the practice of evidence-based medicine. Several tools and guidelines are available for critiquing and assessing the quality of specific study types. However, limited guidance exists for critical appraisal of clinical pharmacokinetic studies. AIM: We aimed to achieve experts' consensus regarding the quality markers for clinical pharmacokinetic studies in an attempt to develop a critical appraisal tool. METHOD: Quality markers related to clinical pharmacokinetic studies, were derived from the published literature and categorized according to manuscript reporting domains (abstract, introduction/background, methodology, results, discussion, and conclusion). Questions that aid in appraising pharmacokinetic studies were formulated from these quality markers. Experts were involved in a modified Delphi process to achieve a consensus regarding the formulated questions. The proposed tool was pilot tested on 30 recently published clinical pharmacokinetic studies. Inter-observer agreement was measured to determine the reliability of the included items. RESULTS: Twenty-five experts consented to participate. Three rounds of a modified Delphi survey were required to generate a consensus for a 21-item tool aimed at appraising the quality of clinical pharmacokinetic studies. When applied to 30 recently published clinical pharmacokinetic studies, most items scored fair to moderate levels of agreement (61.90-95.24%). CONCLUSION: The clinical pharmacokinetic critical appraisal tool (CACPK) developed in this study consisted of 21 items aimed at helping an end-user to determine the quality of a pharmacokinetic study. Further studies are warranted to reaffirm the validity and reliability of the CACPK tool.

Content validation of a Critical Appraisal Tool for Reviewing Analgesia Studies (CATRAS) involving subjects incapable of self-reporting pain.

INTRODUCTION: This article reports the content validation of a Critical Appraisal Tool designed to Review the quality of Analgesia Studies (CATRAS) involving subjects incapable of self-reporting pain and provide guidance as to the strengths and weakness of findings. The CATRAS quality items encompass 3 domains: level of evidence, methodological soundness, and grading of the pain assessment tool. OBJECTIVES: To validate a critical appraisal tool for reviewing analgesia studies involving subjects incapable of self-reporting pain. METHODS: Content validation was achieved using Delphi methodology through panel consensus. A panel of 6 experts reviewed the CATRAS in 3 rounds and quantitatively rated the relevance of the instrument and each of its quality items to their respective domains. RESULTS: Content validation was achieved for each item of the CATRAS and the tool as a whole. Item-level content validity index and kappa coefficient were at least greater than 0.83 and 0.81, respectively, for all items except for one item in domain 2 that was later removed. Scale-level content validity index was 97% (excellent content validity). CONCLUSIONS: This 67-item critical appraisal tool may enable critical and quantitative assessment of the quality of individual analgesia trials involving subjects incapable of self-reporting pain for use in systematic reviews and meta-analysis studies.