RESUMO
OBJECTIVE: Predicting adverse outcomes in patients with peripheral arterial disease (PAD) is a complex task owing to the heterogeneity in patient and disease characteristics. This systematic review aimed to identify prognostic factors and prognostic models to predict mortality outcomes in patients with PAD Fontaine stage I - III or Rutherford category 0 - 4. DATA SOURCES: PubMed, Embase, and Cochrane Database of Systematic Reviews were searched to identify studies examining individual prognostic factors or studies aiming to develop or validate a prognostic model for mortality outcomes in patients with PAD. REVIEW METHODS: Information on study design, patient population, prognostic factors, and prognostic model characteristics was extracted, and risk of bias was evaluated. RESULTS: Sixty nine studies investigated prognostic factors for mortality outcomes in PAD. Over 80 single prognostic factors were identified, with age as a predictor of death in most of the studies. Other common factors included sex, diabetes, and smoking status. Six studies had low risk of bias in all domains, and the remainder had an unclear or high risk of bias in at least one domain. Eight studies developed or validated a prognostic model. All models included age in their primary model, but not sex. All studies had similar discrimination levels of > 70%. Five of the studies on prognostic models had an overall high risk of bias, whereas two studies had an overall unclear risk of bias. CONCLUSION: This systematic review shows that a large number of prognostic studies have been published, with heterogeneity in patient populations, outcomes, and risk of bias. Factors such as sex, age, diabetes, hypertension, and smoking are significant in predicting mortality risk among patients with PAD Fontaine stage I - III or Rutherford category 0 - 4.
RESUMO
BACKGROUND AND PURPOSE: The net benefit of carotid endarterectomy (CEA) is determined partly by the risk of procedural stroke or death. Current guidelines recommend CEA if 30-day risks are <6% for symptomatic stenosis and <3% for asymptomatic stenosis. We aimed to identify prediction models for procedural stroke or death after CEA and to externally validate these models in a large registry of patients from the United States. METHODS: We conducted a systematic search in MEDLINE and EMBASE for prediction models of procedural outcomes after CEA. We validated these models with data from patients who underwent CEA in the American College of Surgeons National Surgical Quality Improvement Program (2011-2017). We assessed discrimination using C statistics and calibration graphically. We determined the number of patients with predicted risks that exceeded recommended thresholds of procedural risks to perform CEA. RESULTS: After screening 788 reports, 15 studies describing 17 prediction models were included. Nine were developed in populations including both asymptomatic and symptomatic patients, 2 in symptomatic and 5 in asymptomatic populations. In the external validation cohort of 26 293 patients who underwent CEA, 702 (2.7%) developed a stroke or died within 30-days. C statistics varied between 0.52 and 0.64 using all patients, between 0.51 and 0.59 using symptomatic patients, and between 0.49 to 0.58 using asymptomatic patients. The Ontario Carotid Endarterectomy Registry model that included symptomatic status, diabetes, heart failure, and contralateral occlusion as predictors, had C statistic of 0.64 and the best concordance between predicted and observed risks. This model identified 4.5% of symptomatic and 2.1% of asymptomatic patients with procedural risks that exceeded recommended thresholds. CONCLUSIONS: Of the 17 externally validated prediction models, the Ontario Carotid Endarterectomy Registry risk model had most reliable predictions of procedural stroke or death after CEA and can inform patients about procedural hazards and help focus CEA toward patients who would benefit most from it.
Assuntos
Estenose das Carótidas/cirurgia , Ensaios Clínicos como Assunto/normas , Endarterectomia das Carótidas/normas , Modelos Teóricos , Seleção de Pacientes , Sistema de Registros/normas , Estenose das Carótidas/diagnóstico , Endarterectomia das Carótidas/métodos , Humanos , Valor Preditivo dos Testes , Medição de Risco/métodos , Medição de Risco/normasRESUMO
BACKGROUND: Describe and evaluate the methodological conduct of prognostic prediction models developed using machine learning methods in oncology. METHODS: We conducted a systematic review in MEDLINE and Embase between 01/01/2019 and 05/09/2019, for studies developing a prognostic prediction model using machine learning methods in oncology. We used the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) statement, Prediction model Risk Of Bias ASsessment Tool (PROBAST) and CHecklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies (CHARMS) to assess the methodological conduct of included publications. Results were summarised by modelling type: regression-, non-regression-based and ensemble machine learning models. RESULTS: Sixty-two publications met inclusion criteria developing 152 models across all publications. Forty-two models were regression-based, 71 were non-regression-based and 39 were ensemble models. A median of 647 individuals (IQR: 203 to 4059) and 195 events (IQR: 38 to 1269) were used for model development, and 553 individuals (IQR: 69 to 3069) and 50 events (IQR: 17.5 to 326.5) for model validation. A higher number of events per predictor was used for developing regression-based models (median: 8, IQR: 7.1 to 23.5), compared to alternative machine learning (median: 3.4, IQR: 1.1 to 19.1) and ensemble models (median: 1.7, IQR: 1.1 to 6). Sample size was rarely justified (n = 5/62; 8%). Some or all continuous predictors were categorised before modelling in 24 studies (39%). 46% (n = 24/62) of models reporting predictor selection before modelling used univariable analyses, and common method across all modelling types. Ten out of 24 models for time-to-event outcomes accounted for censoring (42%). A split sample approach was the most popular method for internal validation (n = 25/62, 40%). Calibration was reported in 11 studies. Less than half of models were reported or made available. CONCLUSIONS: The methodological conduct of machine learning based clinical prediction models is poor. Guidance is urgently needed, with increased awareness and education of minimum prediction modelling standards. Particular focus is needed on sample size estimation, development and validation analysis methods, and ensuring the model is available for independent validation, to improve quality of machine learning based clinical prediction models.
Assuntos
Aprendizado de Máquina , Oncologia , Projetos de Pesquisa , Viés , Humanos , PrognósticoRESUMO
BACKGROUND: While many studies have consistently found incomplete reporting of regression-based prediction model studies, evidence is lacking for machine learning-based prediction model studies. We aim to systematically review the adherence of Machine Learning (ML)-based prediction model studies to the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) Statement. METHODS: We included articles reporting on development or external validation of a multivariable prediction model (either diagnostic or prognostic) developed using supervised ML for individualized predictions across all medical fields. We searched PubMed from 1 January 2018 to 31 December 2019. Data extraction was performed using the 22-item checklist for reporting of prediction model studies ( www.TRIPOD-statement.org ). We measured the overall adherence per article and per TRIPOD item. RESULTS: Our search identified 24,814 articles, of which 152 articles were included: 94 (61.8%) prognostic and 58 (38.2%) diagnostic prediction model studies. Overall, articles adhered to a median of 38.7% (IQR 31.0-46.4%) of TRIPOD items. No article fully adhered to complete reporting of the abstract and very few reported the flow of participants (3.9%, 95% CI 1.8 to 8.3), appropriate title (4.6%, 95% CI 2.2 to 9.2), blinding of predictors (4.6%, 95% CI 2.2 to 9.2), model specification (5.2%, 95% CI 2.4 to 10.8), and model's predictive performance (5.9%, 95% CI 3.1 to 10.9). There was often complete reporting of source of data (98.0%, 95% CI 94.4 to 99.3) and interpretation of the results (94.7%, 95% CI 90.0 to 97.3). CONCLUSION: Similar to prediction model studies developed using conventional regression-based techniques, the completeness of reporting is poor. Essential information to decide to use the model (i.e. model specification and its performance) is rarely reported. However, some items and sub-items of TRIPOD might be less suitable for ML-based prediction model studies and thus, TRIPOD requires extensions. Overall, there is an urgent need to improve the reporting quality and usability of research to avoid research waste. SYSTEMATIC REVIEW REGISTRATION: PROSPERO, CRD42019161764.
Assuntos
Lista de Checagem , Modelos Estatísticos , Humanos , Aprendizado de Máquina , Prognóstico , Aprendizado de Máquina SupervisionadoRESUMO
Clear and informative reporting in titles and abstracts is essential to help readers and reviewers identify potentially relevant studies and decide whether to read the full text. Although the TRIPOD (Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis) statement provides general recommendations for reporting titles and abstracts, more detailed guidance seems to be desirable. The authors present TRIPOD for Abstracts, a checklist and corresponding guidance for reporting prediction model studies in abstracts. A list of 32 potentially relevant items was the starting point for a modified Delphi procedure involving 110 experts, of whom 71 (65%) participated in the web-based survey. After 2 Delphi rounds, the experts agreed on 21 items as being essential to report in abstracts of prediction model studies. This number was reduced by merging some of the items. In a third round, participants provided feedback on a draft version of TRIPOD for Abstracts. The final checklist contains 12 items and applies to journal and conference abstracts that describe the development or external validation of a diagnostic or prognostic prediction model, or the added value of predictors to an existing model, regardless of the clinical domain or statistical approach used.
RESUMO
BACKGROUND: As complete reporting is essential to judge the validity and applicability of multivariable prediction models, a guideline for the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) was introduced. We assessed the completeness of reporting of prediction model studies published just before the introduction of the TRIPOD statement, to refine and tailor its implementation strategy. METHODS: Within each of 37 clinical domains, 10 journals with the highest journal impact factor were selected. A PubMed search was performed to identify prediction model studies published before the launch of TRIPOD in these journals (May 2014). Eligible publications reported on the development or external validation of a multivariable prediction model (either diagnostic or prognostic) or on the incremental value of adding a predictor to an existing model. RESULTS: We included 146 publications (84% prognostic), from which we assessed 170 models: 73 (43%) on model development, 43 (25%) on external validation, 33 (19%) on incremental value, and 21 (12%) on combined development and external validation of the same model. Overall, publications adhered to a median of 44% (25th-75th percentile 35-52%) of TRIPOD items, with 44% (35-53%) for prognostic and 41% (34-48%) for diagnostic models. TRIPOD items that were completely reported for less than 25% of the models concerned abstract (2%), title (5%), blinding of predictor assessment (6%), comparison of development and validation data (11%), model updating (14%), model performance (14%), model specification (17%), characteristics of participants (21%), model performance measures (methods) (21%), and model-building procedures (24%). Most often reported were TRIPOD items regarding overall interpretation (96%), source of data (95%), and risk groups (90%). CONCLUSIONS: More than half of the items considered essential for transparent reporting were not fully addressed in publications of multivariable prediction model studies. Essential information for using a model in individual risk prediction, i.e. model specifications and model performance, was incomplete for more than 80% of the models. Items that require improved reporting are title, abstract, and model-building procedures, as they are crucial for identification and external validation of prediction models.
Assuntos
Projetos de Pesquisa , Humanos , PrognósticoRESUMO
OBJECTIVES: To assess the endorsement of reporting guidelines by high impact factor journals over the period 2017-2022, with a specific focus on the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) statement. STUDY DESIGN AND SETTING: We searched the online 'instructions to authors' of high impact factor medical journals in February 2017 and in January 2022 for any reference to reporting guidelines and TRIPOD in particular. RESULTS: In 2017, 205 out of 337 (61%) journals mentioned any reporting guideline in their instructions to authors and in 2022 this increased to 245 (73%) journals. A reference to TRIPOD was provided by 27 (8%) journals in 2017 and 67 (20%) in 2022. Of those journals mentioning TRIPOD in 2022, 22% provided a link to the TRIPOD website and 60% linked to TRIPOD information on the Enhancing the QUAlity and Transparency Of health Research (EQUATOR) Network website. Twenty-five percent of the journals required adherence to TRIPOD. CONCLUSION: About three-quarters of high-impact medical journals endorse the use of reporting guidelines and 20% endorse TRIPOD. Transparent reporting is important in enhancing the usefulness of health research and endorsement by journals plays a critical role in this.
Assuntos
Publicações Periódicas como Assunto , Humanos , Prognóstico , Inquéritos e QuestionáriosRESUMO
OBJECTIVES: To develop a framework to identify and evaluate spin practices and its facilitators in studies on clinical prediction model regardless of the modeling technique. STUDY DESIGN AND SETTING: We followed a three-phase consensus process: (1) premeeting literature review to generate items to be included; (2) a series of structured meetings to provide comments discussed and exchanged viewpoints on items to be included with a panel of experienced researchers; and (3) postmeeting review on final list of items and examples to be included. Through this iterative consensus process, a framework was derived after all panel's researchers agreed. RESULTS: This consensus process involved a panel of eight researchers and resulted in SPIN-Prediction Models which consists of two categories of spin (misleading interpretation and misleading transportability), and within these categories, two forms of spin (spin practices and facilitators of spin). We provide criteria and examples. CONCLUSION: We proposed this guidance aiming to facilitate not only the accurate reporting but also an accurate interpretation and extrapolation of clinical prediction models which will likely improve the reporting quality of subsequent research, as well as reduce research waste.
Assuntos
Consenso , Humanos , Projetos de Pesquisa/normas , Modelos EstatísticosRESUMO
OBJECTIVES: To determine whether clinical trial register (CTR) searches can accurately identify a greater number of completed randomized clinical trials (RCTs) than electronic bibliographic database (EBD) searches for systematic reviews of interventions, and to quantify the number of eligible ongoing trials. STUDY DESIGN AND SETTING: We performed an evaluation study and based our search for RCTs on the eligibility criteria of a systematic review that focused on the underrepresentation of people with chronic kidney disease in cardiovascular RCTs. We conducted a combined search of ClinicalTrials.gov and the WHO International Clinical Trials Registry Platform through the Cochrane Central Register of Controlled Trials to identify eligible RCTs registered up to June 1, 2023. We searched Cochrane Central Register of Controlled Trials, EMBASE, and MEDLINE for publications of eligible RCTs published up to June 5, 2023. Finally, we compared the search results to determine the extent to which the two sources identified the same RCTs. RESULTS: We included 92 completed RCTs. Of these, 81 had results available. Sixty-six completed RCTs with available results were identified by both sources (81% agreement [95% CI: 71-88]). We identified seven completed RCTs with results exclusively by CTR search (9% [95% CI: 4-17]) and eight exclusively by EBD search (10% [95% CI: 5-18]). Eleven RCTs were completed but lacked results (four identified by both sources (36% [95% CI: 15-65]), one exclusively by EBD search (9% [95% CI: 1-38]), and six exclusively by CTR search (55% [95% CI: 28-79])). Also, we identified 42 eligible ongoing RCTs: 16 by both sources (38% [95% CI: 25-53]) and 26 exclusively by CTR search (62% [95% CI: 47-75]). Lastly, we identified four RCTs of unknown status by both sources. CONCLUSION: CTR searches identify a greater number of completed RCTs than EBD searches. Both searches missed some included RCTs. Based on our case study, researchers (eg, information specialists, systematic reviewers) aiming to identify all available RCTs should continue to search both sources. Once the barriers to performing CTR searches alone are targeted, CTR searches may be a suitable alternative.
Assuntos
Bases de Dados Bibliográficas , Ensaios Clínicos Controlados Aleatórios como Assunto , Sistema de Registros , Revisões Sistemáticas como Assunto , Ensaios Clínicos Controlados Aleatórios como Assunto/estatística & dados numéricos , Ensaios Clínicos Controlados Aleatórios como Assunto/normas , Ensaios Clínicos Controlados Aleatórios como Assunto/métodos , Humanos , Revisões Sistemáticas como Assunto/métodos , Bases de Dados Bibliográficas/estatística & dados numéricos , Sistema de Registros/estatística & dados numéricos , Armazenamento e Recuperação da Informação/métodos , Armazenamento e Recuperação da Informação/estatística & dados numéricosRESUMO
OBJECTIVES: Risk of bias assessments are important in meta-analyses of both aggregate and individual participant data (IPD). There is limited evidence on whether and how risk of bias of included studies or datasets in IPD meta-analyses (IPDMAs) is assessed. We review how risk of bias is currently assessed, reported, and incorporated in IPDMAs of test accuracy and clinical prediction model studies and provide recommendations for improvement. STUDY DESIGN AND SETTING: We searched PubMed (January 2018-May 2020) to identify IPDMAs of test accuracy and prediction models, then elicited whether each IPDMA assessed risk of bias of included studies and, if so, how assessments were reported and subsequently incorporated into the IPDMAs. RESULTS: Forty-nine IPDMAs were included. Nineteen of 27 (70%) test accuracy IPDMAs assessed risk of bias, compared to 5 of 22 (23%) prediction model IPDMAs. Seventeen of 19 (89%) test accuracy IPDMAs used Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2), but no tool was used consistently among prediction model IPDMAs. Of IPDMAs assessing risk of bias, 7 (37%) test accuracy IPDMAs and 1 (20%) prediction model IPDMA provided details on the information sources (e.g., the original manuscript, IPD, primary investigators) used to inform judgments, and 4 (21%) test accuracy IPDMAs and 1 (20%) prediction model IPDMA provided information or whether assessments were done before or after obtaining the IPD of the included studies or datasets. Of all included IPDMAs, only seven test accuracy IPDMAs (26%) and one prediction model IPDMA (5%) incorporated risk of bias assessments into their meta-analyses. For future IPDMA projects, we provide guidance on how to adapt tools such as Prediction model Risk Of Bias ASsessment Tool (for prediction models) and QUADAS-2 (for test accuracy) to assess risk of bias of included primary studies and their IPD. CONCLUSION: Risk of bias assessments and their reporting need to be improved in IPDMAs of test accuracy and, especially, prediction model studies. Using recommended tools, both before and after IPD are obtained, will address this.
Assuntos
Confiabilidade dos Dados , Modelos Estatísticos , Humanos , Prognóstico , ViésRESUMO
BACKGROUND: Prognostic models are typically developed to estimate the risk that an individual in a particular health state will develop a particular health outcome, to support (shared) decision making. Systematic reviews of prognostic model studies can help identify prognostic models that need to further be validated or are ready to be implemented in healthcare. OBJECTIVES: To provide a step-by-step guidance on how to conduct and read a systematic review of prognostic model studies and to provide an overview of methodology and guidance available for every step of the review progress. SOURCES: Published, peer-reviewed guidance articles. CONTENT: We describe the following steps for conducting a systematic review of prognosis studies: 1) Developing the review question using the Population, Index model, Comparator model, Outcome(s), Timing, Setting format, 2) Searching and selection of articles, 3) Data extraction using the Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies (CHARMS) checklist, 4) Quality and risk of bias assessment using the Prediction model Risk Of Bias ASsessment (PROBAST) tool, 5) Analysing data and undertaking quantitative meta-analysis, and 6) Presenting summary of findings, interpreting results, and drawing conclusions. Guidance for each step is described and illustrated using a case study on prognostic models for patients with COVID-19. IMPLICATIONS: Guidance for conducting a systematic review of prognosis studies is available, but the implications of these reviews for clinical practice and further research highly depend on complete reporting of primary studies.
Assuntos
COVID-19 , Humanos , Prognóstico , ViésRESUMO
BACKGROUND: Navigation bronchoscopy has seen rapid development in the past decade in terms of new navigation techniques and multi-modality approaches utilizing different techniques and tools. This systematic review analyses the diagnostic yield and safety of navigation bronchoscopy for the diagnosis of peripheral pulmonary nodules suspected of lung cancer. METHODS: An extensive search was performed in Embase, Medline and Cochrane CENTRAL in May 2022. Eligible studies used cone-beam CT-guided navigation (CBCT), electromagnetic navigation (EMN), robotic navigation (RB) or virtual bronchoscopy (VB) as the primary navigation technique. Primary outcomes were diagnostic yield and adverse events. Quality of studies was assessed using QUADAS-2. Random effects meta-analysis was performed, with subgroup analyses for different navigation techniques, newer versus older techniques, nodule size, publication year, and strictness of diagnostic yield definition. Explorative analyses of subgroups reported by studies was performed for nodule size and bronchus sign. RESULTS: A total of 95 studies (n = 10,381 patients; n = 10,682 nodules) were included. The majority (n = 63; 66.3%) had high risk of bias or applicability concerns in at least one QUADAS-2 domain. Summary diagnostic yield was 70.9% (95%-CI 68.4%-73.2%). Overall pneumothorax rate was 2.5%. Newer navigation techniques using advanced imaging and/or robotics(CBCT, RB, tomosynthesis guided EMN; n = 24 studies) had a statistically significant higher diagnostic yield compared to longer established techniques (EMN, VB; n = 82 studies): 77.5% (95%-CI 74.7%-80.1%) vs 68.8% (95%-CI 65.9%-71.6%) (p < 0.001).Explorative subgroup analyses showed that larger nodule size and bronchus sign presence were associated with a statistically significant higher diagnostic yield. Other subgroup analyses showed no significant differences. CONCLUSION: Navigation bronchoscopy is a safe procedure, with the potential for high diagnostic yield, in particular using newer techniques such as RB, CBCT and tomosynthesis-guided EMN. Studies showed a large amount of heterogeneity, making comparisons difficult. Standardized definitions for outcomes with relevant clinical context will improve future comparability.
Assuntos
Neoplasias Pulmonares , Nódulo Pulmonar Solitário , Humanos , Broncoscopia/efeitos adversos , Broncoscopia/métodos , Neoplasias Pulmonares/diagnóstico , Neoplasias Pulmonares/etiologia , Nódulo Pulmonar Solitário/diagnóstico por imagem , Brônquios , Tomografia Computadorizada de Feixe CônicoRESUMO
INTRODUCTION: Patients with chronic kidney disease (CKD) have a high risk of cardiovascular disease (CVD). Prediction models, combining clinical and laboratory characteristics, are commonly used to estimate an individual's CVD risk. However, these models are not specifically developed for patients with CKD and may therefore be less accurate. In this review, we aim to give an overview of CVD prognostic studies available, and their methodological quality, specifically for patients with CKD. METHODS: MEDLINE was searched for papers reporting CVD prognostic studies in patients with CKD published between 2012 and 2021. Characteristics regarding patients, study design, outcome measurement, and prediction models were compared between included studies. The risk of bias of studies reporting on prognostic factors or the development/validation of a prediction model was assessed with, respectively, the QUIPS and PROBAST tool. RESULTS: In total, 134 studies were included, of which 123 studies tested the incremental value of one or more predictors to existing models or common risk factors, while only 11 studies reported on the development or validation of a prediction model. Substantial heterogeneity in cohort and study characteristics, such as sample size, event rate, and definition of outcome measurements, was observed across studies. The most common predictors were age (87%), sex (75%), diabetes (70%), and estimated glomerular filtration rate (69%). Most of the studies on prognostic factors have methodological shortcomings, mostly due to a lack of reporting on clinical and methodological information. Of the 11 studies on prediction models, six developed and internally validated a model and four externally validated existing or developed models. Only one study on prognostic models showed a low risk of bias and high applicability. CONCLUSION: A large quantity of prognostic studies has been published, yet their usefulness remains unclear due to incomplete presentation, and lack of external validation of prognostic models. Our review can be used to select the most appropriate prognostic model depending on the patient population, outcome, and risk of bias. Future collaborative efforts should aim at improving existing models by externally validating them, evaluating the addition of new predictors, and assessment of the clinical impact. REGISTRATION: We have registered the protocol of our systematic review on PROSPERO (CRD42021228043).
Assuntos
Doenças Cardiovasculares , Insuficiência Renal Crônica , Humanos , Doenças Cardiovasculares/complicações , Doenças Cardiovasculares/epidemiologia , Frequência Cardíaca , Prognóstico , Insuficiência Renal Crônica/complicações , Fatores de RiscoRESUMO
BACKGROUND AND OBJECTIVES: We sought to summarize the study design, modelling strategies, and performance measures reported in studies on clinical prediction models developed using machine learning techniques. METHODS: We search PubMed for articles published between 01/01/2018 and 31/12/2019, describing the development or the development with external validation of a multivariable prediction model using any supervised machine learning technique. No restrictions were made based on study design, data source, or predicted patient-related health outcomes. RESULTS: We included 152 studies, 58 (38.2% [95% CI 30.8-46.1]) were diagnostic and 94 (61.8% [95% CI 53.9-69.2]) prognostic studies. Most studies reported only the development of prediction models (n = 133, 87.5% [95% CI 81.3-91.8]), focused on binary outcomes (n = 131, 86.2% [95% CI 79.8-90.8), and did not report a sample size calculation (n = 125, 82.2% [95% CI 75.4-87.5]). The most common algorithms used were support vector machine (n = 86/522, 16.5% [95% CI 13.5-19.9]) and random forest (n = 73/522, 14% [95% CI 11.3-17.2]). Values for area under the Receiver Operating Characteristic curve ranged from 0.45 to 1.00. Calibration metrics were often missed (n = 494/522, 94.6% [95% CI 92.4-96.3]). CONCLUSION: Our review revealed that focus is required on handling of missing values, methods for internal validation, and reporting of calibration to improve the methodological conduct of studies on machine learning-based prediction models. SYSTEMATIC REVIEW REGISTRATION: PROSPERO, CRD42019161764.
Assuntos
Aprendizado de Máquina , Aprendizado de Máquina Supervisionado , Humanos , Algoritmos , Prognóstico , Curva ROCRESUMO
OBJECTIVES: In biomedical research, spin is the overinterpretation of findings, and it is a growing concern. To date, the presence of spin has not been evaluated in prognostic model research in oncology, including studies developing and validating models for individualized risk prediction. STUDY DESIGN AND SETTING: We conducted a systematic review, searching MEDLINE and EMBASE for oncology-related studies that developed and validated a prognostic model using machine learning published between 1st January, 2019, and 5th September, 2019. We used existing spin frameworks and described areas of highly suggestive spin practices. RESULTS: We included 62 publications (including 152 developed models; 37 validated models). Reporting was inconsistent between methods and the results in 27% of studies due to additional analysis and selective reporting. Thirty-two studies (out of 36 applicable studies) reported comparisons between developed models in their discussion and predominantly used discrimination measures to support their claims (78%). Thirty-five studies (56%) used an overly strong or leading word in their title, abstract, results, discussion, or conclusion. CONCLUSION: The potential for spin needs to be considered when reading, interpreting, and using studies that developed and validated prognostic models in oncology. Researchers should carefully report their prognostic model research using words that reflect their actual results and strength of evidence.
Assuntos
Oncologia , Pesquisa , Humanos , Prognóstico , Aprendizado de MáquinaRESUMO
OBJECTIVES: We evaluated the presence and frequency of spin practices and poor reporting standards in studies that developed and/or validated clinical prediction models using supervised machine learning techniques. STUDY DESIGN AND SETTING: We systematically searched PubMed from 01/2018 to 12/2019 to identify diagnostic and prognostic prediction model studies using supervised machine learning. No restrictions were placed on data source, outcome, or clinical specialty. RESULTS: We included 152 studies: 38% reported diagnostic models and 62% prognostic models. When reported, discrimination was described without precision estimates in 53/71 abstracts (74.6% [95% CI 63.4-83.3]) and 53/81 main texts (65.4% [95% CI 54.6-74.9]). Of the 21 abstracts that recommended the model to be used in daily practice, 20 (95.2% [95% CI 77.3-99.8]) lacked any external validation of the developed models. Likewise, 74/133 (55.6% [95% CI 47.2-63.8]) studies made recommendations for clinical use in their main text without any external validation. Reporting guidelines were cited in 13/152 (8.6% [95% CI 5.1-14.1]) studies. CONCLUSION: Spin practices and poor reporting standards are also present in studies on prediction models using machine learning techniques. A tailored framework for the identification of spin will enhance the sound reporting of prediction model studies.
Assuntos
Aprendizado de Máquina , Humanos , PrognósticoRESUMO
The aim of this paper is to summarize all available evidence from systematic reviews, randomized controlled trials (RCTs) and comparative nonrandomized studies (NRS) on the association between nutrition and antioxidant, vitamin, and mineral supplements and the development or progression of age-related macular degeneration (AMD). The Cochrane Database of Systematic Reviews, Cochrane register CENTRAL, MEDLINE and Embase were searched and studies published between January 2015 and May 2021 were included. The certainty of evidence was assessed according to the GRADE methodology. The main outcome measures were development of AMD, progression of AMD, and side effects. We included 7 systematic reviews, 7 RCTs, and 13 NRS. A high consumption of specific nutrients, i.e. ß-carotene, lutein and zeaxanthin, copper, folate, magnesium, vitamin A, niacin, vitamin B6, vitamin C, docosahexaenoic acid, and eicosapentaenoic acid, was associated with a lower risk of progression of early to late AMD (high certainty of evidence). Use of antioxidant supplements and adherence to a Mediterranean diet, characterized by a high consumption of vegetables, whole grains, and nuts and a low consumption of red meat, were associated with a decreased risk of progression of early to late AMD (moderate certainty of evidence). A high consumption of alcohol was associated with a higher risk of developing AMD (moderate certainty of evidence). Supplementary vitamin C, vitamin E, or ß-carotene were not associated with the development of AMD, and supplementary omega-3 fatty acids were not associated with progression to late AMD (high certainty of evidence). Research in the last 35 years included in our overview supports that a high intake of specific nutrients, the use of antioxidant supplements and adherence to a Mediterranean diet decrease the risk of progression of early to late AMD.
Assuntos
Antioxidantes , Degeneração Macular , Humanos , Antioxidantes/uso terapêutico , Ácido Ascórbico/uso terapêutico , beta Caroteno/uso terapêutico , Suplementos Nutricionais , Degeneração Macular/etiologia , Degeneração Macular/prevenção & controle , Degeneração Macular/tratamento farmacológico , VitaminasRESUMO
BACKGROUND: With the exponential growth of published systematic reviews (SR), there is a high potential for overlapping and redundant duplication of work. Prospective protocol registration gives the opportunity to assess the added value of a new study or review, thereby potentially reducing research waste and simultaneously increasing transparency and research quality. The PROSPERO database for SR protocol registration was launched 10 years ago. This study aims to assess the proportion SRs of intervention studies with a protocol registration (or publication) and explore associations of SR characteristics with protocol registration status. METHODS: PubMed was searched for SRs of human intervention studies published in January 2020 and January 2021. After random-stratified sampling and eligibility screening, data extraction on publication and journal characteristics, and protocol registration status, was performed. Both descriptive and multivariable comparative statistical analyses were performed. RESULTS: A total of 357 SRs (2020: n = 163; 2021: n = 194) were included from a random sample of 1267 publications. Of the published SRs, 38% had a protocol. SRs that reported using PRISMA as a reporting guideline had higher odds of having a protocol than publications that did not report PRISMA (OR 2.71; 95% CI: 1.21 to 6.09). SRs with a higher journal impact factor had higher odds of having a protocol (OR 1.12; 95% CI 1.04 to 1.25). Publications from Asia had a lower odds of having a protocol (OR 0.43; 95% CI 0.23 to 0.80, reference category = Europe). Of the 33 SRs published in journals that endorse PROSPERO, 45% did not have a protocol. Most SR protocols were registered in PROSPERO (n = 129; 96%). CONCLUSIONS: We found that 38% of recently published SRs of interventions reported a registered or published protocol. Protocol registration was significantly associated with a higher impact factor of the journal publishing the SR and a more frequent self-reported use of the PRISMA guidelines. In some parts of the world, SR protocols are more often registered or published than others. To guide strategies to increase the uptake of SR protocol registration, further research is needed to gain understanding of the benefits and informativeness of SRs protocols among different stakeholders. SYSTEMATIC REVIEW REGISTRATION: osf.io/9kj7r/.
Assuntos
Relatório de Pesquisa , Revisões Sistemáticas como Assunto , Ásia , Humanos , Fator de Impacto de Revistas , Estudos Prospectivos , Projetos de PesquisaRESUMO
Background: Current cardiovascular risk assessment in people living with HIV is based on general risk assessment tools; however, whether these tools can be applied in sub-Saharan African populations has been questioned. Objectives: The study aimed to assess cardiovascular risk classification of common cardiovascular disease (CVD) risk prediction models compared to the Data Collection on Adverse Events of Anti-HIV Drugs (D:A:D) 2010 and 2016 models in people living with HIV. Method: Cardiovascular disease risk was estimated by Framingham Cardiovascular and Heart Disease (FHS-CVD, FHS-CHD), Atherosclerotic Cardiovascular Disease (ASCVD) and D:A:D 2010 and 2016 risk prediction models for HIV-infected participants of the Ndlovu Cohort Study, Limpopo, rural South Africa. Participants were classified to be at low (< 10%), moderate (10% - 20%), or high-risk (> 20%) of CVD within 10 years for general CVD and five years for D:A:D models. Kappa statistics were used to determine agreement between CVD risk prediction models. Subgroup analysis was performed according to age. Results: The analysis comprised 735 HIV-infected individuals, predominantly women (56.7%), average age 43.9 (8.8) years. The median predicted CVD risk for D:A:D 2010 and FHS-CVD was 4% and for ASCVD and FHS-CHD models, 3%. For the D:A:D 2016 risk prediction model, the figure was 5%. High 10-year CVD risk was predicted for 2.9%, 0.5%, 0.7%, 3.1% and 6.6% of the study participants by FHS-CVD, FHS-CHD, ASCVD, and D:A:D 2010 and 2016. Kappa statistics ranged from 0.34 for ASCVD to 0.60 for FHS-CVD as compared to the D:A:D 2010 risk prediction model. Conclusion: Overall, predicted CVD risk is low in this population. Compared to D:A:D 2010, CVD risk estimated by the FHS-CVD model showed similar overall results for risk classification. With the exception of the D:A:D model, all other risk prediction models classified fewer people to be at high estimated CVD risk. Prospective studies are needed to develop and validate CVD risk algorithms in people living with HIV in sub-Saharan Africa.