A literature review of quality assessment and applicability to HTA of risk prediction models of coronary heart disease in patients with diabetes.

This literature review had two objectives: to identify models for predicting the risk of coronary heart diseases in patients with diabetes (DM); and to assess model quality in terms of risk of bias (RoB) and applicability for the purpose of health technology assessment (HTA). We undertook a targeted review of journal articles published in English, Dutch, Chinese, or Spanish in 5 databases from 1st January 2016 to 18th December 2022, and searched three systematic reviews for the models published after 2012. We used PROBAST (Prediction model Risk Of Bias Assessment Tool) to assess RoB, and used findings from Betts et al. 2019, which summarized recommendations and criticisms of HTA agencies on cardiovascular risk prediction models, to assess model applicability for the purpose of HTA. As a result, 71 % and 67 % models reporting C-index showed good discrimination abilities (C-index >= 0.7). Of the 26 model studies and 30 models identified, only one model study showed low RoB in all domains, and no model was fully applicable for HTA. Since the major cause of high RoB is inappropriate use of analysis method, we advise clinicians to carefully examine the model performance declared by model developers, and to trust a model if all PROBAST domains except analysis show low RoB and at least one validation study conducted in the same setting (e.g. country) is available. Moreover, since general model applicability is not informative for HTA, novel adapted tools may need to be developed.

Tools for assessing quality of studies investigating health interventions using real-world data: a literature review and content analysis.

OBJECTIVES: We aimed to identify existing appraisal tools for non-randomised studies of interventions (NRSIs) and to compare the criteria that the tools provide at the quality-item level. DESIGN: Literature review through three approaches: systematic search of journal articles, snowballing search of reviews on appraisal tools and grey literature search on websites of health technology assessment (HTA) agencies. DATA SOURCES: Systematic search: Medline; Snowballing: starting from three articles (D'Andrea et al, Quigley et al and Faria et al); Grey literature: websites of European HTA agencies listed by the International Network of Agencies for Health Technology Assessment. Appraisal tools were searched through April 2022. ELIGIBILITY CRITERIA FOR SELECTING STUDIES: We included a tool, if it addressed quality concerns of NRSIs and was published in English (unless from grey literature). A tool was excluded, if it was only for diagnostic, prognostic, qualitative or secondary studies. DATA EXTRACTION AND SYNTHESIS: Two independent researchers searched, screened and reviewed all included studies and tools, summarised quality items and scored whether and to what extent a quality item was described by a tool, for either methodological quality or reporting. RESULTS: Forty-nine tools met inclusion criteria and were included for the content analysis. Concerns regarding the quality of NRSI were categorised into 4 domains and 26 items. The Research Triangle Institute Item Bank (RTI Item Bank) and STrengthening the Reporting of OBservational studies in Epidemiology (STROBE) were the most comprehensive tools for methodological quality and reporting, respectively, as they addressed (n=20; 17) and sufficiently described (n=18; 13) the highest number of items. However, none of the tools covered all items. CONCLUSION: Most of the tools have their own strengths, but none of them could address all quality concerns relevant to NRSIs. Even the most comprehensive tools can be complemented by several items. We suggest decision-makers, researchers and tool developers consider the quality-item level heterogeneity, when selecting a tool or identifying a research gap. OSF REGISTRATION NUMBER: OSF registration DOI (https://doi.org/10.17605/OSF.IO/KCSGX).

Understanding innovation of health technology assessment methods: the IHTAM framework.

Adequate methods are urgently needed to guarantee the good practice of health technology assessment (HTA) for technologies with novel properties. The aim of the study was to construct a conceptual framework to help understand the innovation of HTA methods (IHTAM). The construction of the IHTAM framework was based on two scoping reviews, one on the current practice of innovating methods, that is existing HTA frameworks, and one on theoretical foundations for innovating methods outside the HTA discipline. Both aimed to identify and synthesize concepts of innovation (i.e., innovation processes and roles of stakeholders in innovation). Using these concepts, the framework was developed in iterative brainstorming sessions and subsequent discussions with representatives from various stakeholder groups. The framework was constructed based on twenty documents on innovating HTA frameworks and fourteen guidelines from three scientific disciplines. It includes a generic innovation process consisting of three phases ("Identification," "Development," and "Implementation") and nine subphases. In the framework, three roles that HTA stakeholders can play in innovation ("Developers," "Practitioners," and "Beneficiaries") are defined, and a process on how the stakeholders innovate HTA methods is included. The IHTAM framework visualizes systematically which elements and stakeholders are important to the development and implementation of novel HTA methods. The framework could be used by all stakeholders involved in HTA innovation to learn how to engage dynamically and collaborate effectively throughout the innovation process. HTA stakeholders in practice have welcomed the framework, though additional testing of its applicability and acceptance is essential.

Toward a less costly but accurate calculation of the CCSD(T)/CBS noncovalent interaction energy.

The popular method of calculating the noncovalent interaction energies at the coupled-cluster single-, double-, and perturbative triple-excitations [CCSD(T)] theory level in the complete basis set (CBS) limit was to add a CCSD(T) correction term to the CBS second-order Møller-Plesset perturbation theory (MP2). The CCSD(T) correction term is the difference between the CCSD(T) and MP2 interaction energies evaluated in a medium basis set. However, the CCSD(T) calculations with the medium basis sets are still very expensive for systems with more than 30 atoms. Comparatively, the domain-based local pair natural orbital coupled-cluster method [DLPNO-CCSD(T)] can be applied to large systems with over 1,000 atoms. Considering both the computational accuracy and efficiency, in this work, we propose a new scheme to calculate the CCSD(T)/CBS interaction energies. In this scheme, the MP2/CBS term keeps intact and the CCSD(T) correction term is replaced by a DLPNO-CCSD(T) correction term which is the difference between the DLPNO-CCSD(T) and DLPNO-MP2 interaction energies evaluated in a medium basis set. The interaction energies of the noncovalent systems in the S22, HSG, HBC6, NBC10, and S66 databases were recalculated employing this new scheme. The consistent and tight settings of the truncation parameters for DLPNO-CCSD(T) and DLPNO-MP2 in this noncanonical CCSD(T)/CBS calculations lead to the maximum absolute deviation and root-mean-square deviation from the canonical CCSD(T)/CBS interaction energies of less than or equal to 0.28 kcal/mol and 0.09 kcal/mol, respectively. The high accuracy and low cost of this new computational scheme make it an excellent candidate for the study of large noncovalent systems.