RESUMEN
We present the protocol for a study testing the hypothesis that a computer-aided diagnosis (CAD) system for three-dimensional multiparametric ultrasound (3D mpUS) is noninferior to magnetic resonance imaging (MRI) in guiding prostate biopsies for detection of clinically significant prostate cancer (csPCa). The prospective study has a fully paired design for assessment of diagnostic accuracy and is registered on ClinicalTrials.gov as NCT06281769. A total of 438 biopsy-naïve men scheduled for prostate MRI evaluation because of an abnormal digital rectal examination and/or elevated serum prostate-specific antigen will be included. All patients will undergo both MRI (multiparametric or biparametric) and 3D mpUS with CAD (PCaVision). Suspicious lesions will be independently identified using each imaging technique. MRI targeted biopsy (TBx) and/or PCaVision TBx will be performed if suspicious lesions are identified on imaging. When both PCaVision and MRI identify lesions in an individual patient, the TBx order for this patient will be randomized. Three TBx samples per lesion will be taken for a maximum of two lesions per modality. The primary objective is the detection rate for csPCa (International Society of Urological Pathology grade group [GG] ≥2) with the PCaVision versus the MRI TBx pathway. The noninferiority margin for the absolute difference in detection rates is set at a difference of 5%. Secondary outcomes are the proportion of men in whom TBx could have been safely omitted in each pathway. Additional diagnostic accuracy analyses will be performed for different definitions of PCa (GG ≥3; GG ≥2 with cribriform growth and/or intraductal carcinoma; and GG 1). The frequency of insufficient image quality for the two pathways will also be assessed. Lastly, we will determine the diagnostic performance for csPCa detection at various 3D mpUS image quality thresholds for PCaVision.
RESUMEN
OBJECTIVES: To develop a framework to identify and evaluate spin practices and its facilitators in studies on clinical prediction model regardless of the modeling technique. STUDY DESIGN AND SETTING: We followed a three-phase consensus process: (1) premeeting literature review to generate items to be included; (2) a series of structured meetings to provide comments discussed and exchanged viewpoints on items to be included with a panel of experienced researchers; and (3) postmeeting review on final list of items and examples to be included. Through this iterative consensus process, a framework was derived after all panel's researchers agreed. RESULTS: This consensus process involved a panel of eight researchers and resulted in SPIN-Prediction Models which consists of two categories of spin (misleading interpretation and misleading transportability), and within these categories, two forms of spin (spin practices and facilitators of spin). We provide criteria and examples. CONCLUSION: We proposed this guidance aiming to facilitate not only the accurate reporting but also an accurate interpretation and extrapolation of clinical prediction models which will likely improve the reporting quality of subsequent research, as well as reduce research waste.
Asunto(s)
Consenso , Humanos , Proyectos de Investigación/normas , Modelos EstadísticosRESUMEN
INTRODUCTION: Meta-analyses show postive effects of telemedicine in heart failure (HF) management on hospitalisation, mortality and costs. However, these effects are heterogeneous due to variation in the included HF population, the telemedicine components and the quality of the comparator usual care. Still, telemedicine is gaining acceptance in HF management. The current nationwide study aims to identify (1) in which subgroup(s) of patients with HF telemedicine is (cost-)effective and (2) which components of telemedicine are most (cost-)effective. METHODS AND ANALYSIS: The RELEASE-HF ('REsponsible roLl-out of E-heAlth through Systematic Evaluation - Heart Failure') study is a multicentre, observational, registry-based cohort study that plans to enrol 6480 patients with HF using data from the HF registry facilitated by the Netherlands Heart Registration. Collected data include patient characteristics, treatment information and clinical outcomes, and are measured at HF diagnosis and at 6 and 12 months afterwards. The components of telemedicine are described at the hospital level based on closed-ended interviews with clinicians and at the patient level based on additional data extracted from electronic health records and telemedicine-generated data. The costs of telemedicine are calculated using registration data and interviews with clinicians and finance department staff. To overcome missing data, additional national databases will be linked to the HF registry if feasible. Heterogeneity of the effects of offering telemedicine compared with not offering on days alive without unplanned hospitalisations in 1 year is assessed across predefined patient characteristics using exploratory stratified analyses. The effects of telemedicine components are assessed by fitting separate models for component contrasts. ETHICS AND DISSEMINATION: The study has been approved by the Medical Ethics Committee 2021 of the University Medical Center Utrecht (the Netherlands). Results will be published in peer-reviewed journals and presented at (inter)national conferences. Effective telemedicine scenarios will be proposed among hospitals throughout the country and abroad, if applicable and feasible. TRIAL REGISTRATION NUMBER: NCT05654961.
Asunto(s)
Insuficiencia Cardíaca , Telemedicina , Humanos , Estudios de Cohortes , Países Bajos , Sistema de Registros , Telemedicina/métodos , Estudios Observacionales como AsuntoRESUMEN
OBJECTIVES: Risk of bias assessments are important in meta-analyses of both aggregate and individual participant data (IPD). There is limited evidence on whether and how risk of bias of included studies or datasets in IPD meta-analyses (IPDMAs) is assessed. We review how risk of bias is currently assessed, reported, and incorporated in IPDMAs of test accuracy and clinical prediction model studies and provide recommendations for improvement. STUDY DESIGN AND SETTING: We searched PubMed (January 2018-May 2020) to identify IPDMAs of test accuracy and prediction models, then elicited whether each IPDMA assessed risk of bias of included studies and, if so, how assessments were reported and subsequently incorporated into the IPDMAs. RESULTS: Forty-nine IPDMAs were included. Nineteen of 27 (70%) test accuracy IPDMAs assessed risk of bias, compared to 5 of 22 (23%) prediction model IPDMAs. Seventeen of 19 (89%) test accuracy IPDMAs used Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2), but no tool was used consistently among prediction model IPDMAs. Of IPDMAs assessing risk of bias, 7 (37%) test accuracy IPDMAs and 1 (20%) prediction model IPDMA provided details on the information sources (e.g., the original manuscript, IPD, primary investigators) used to inform judgments, and 4 (21%) test accuracy IPDMAs and 1 (20%) prediction model IPDMA provided information or whether assessments were done before or after obtaining the IPD of the included studies or datasets. Of all included IPDMAs, only seven test accuracy IPDMAs (26%) and one prediction model IPDMA (5%) incorporated risk of bias assessments into their meta-analyses. For future IPDMA projects, we provide guidance on how to adapt tools such as Prediction model Risk Of Bias ASsessment Tool (for prediction models) and QUADAS-2 (for test accuracy) to assess risk of bias of included primary studies and their IPD. CONCLUSION: Risk of bias assessments and their reporting need to be improved in IPDMAs of test accuracy and, especially, prediction model studies. Using recommended tools, both before and after IPD are obtained, will address this.
Asunto(s)
Exactitud de los Datos , Modelos Estadísticos , Humanos , Pronóstico , SesgoRESUMEN
OBJECTIVES: To assess the endorsement of reporting guidelines by high impact factor journals over the period 2017-2022, with a specific focus on the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) statement. STUDY DESIGN AND SETTING: We searched the online 'instructions to authors' of high impact factor medical journals in February 2017 and in January 2022 for any reference to reporting guidelines and TRIPOD in particular. RESULTS: In 2017, 205 out of 337 (61%) journals mentioned any reporting guideline in their instructions to authors and in 2022 this increased to 245 (73%) journals. A reference to TRIPOD was provided by 27 (8%) journals in 2017 and 67 (20%) in 2022. Of those journals mentioning TRIPOD in 2022, 22% provided a link to the TRIPOD website and 60% linked to TRIPOD information on the Enhancing the QUAlity and Transparency Of health Research (EQUATOR) Network website. Twenty-five percent of the journals required adherence to TRIPOD. CONCLUSION: About three-quarters of high-impact medical journals endorse the use of reporting guidelines and 20% endorse TRIPOD. Transparent reporting is important in enhancing the usefulness of health research and endorsement by journals plays a critical role in this.
Asunto(s)
Publicaciones Periódicas como Asunto , Humanos , Pronóstico , Encuestas y CuestionariosRESUMEN
BACKGROUND: A previous individual participant data meta-analysis (IPD-MA) of antibiotics for adults with clinically diagnosed acute rhinosinusitis (ARS) showed a marginal overall effect of antibiotics, but was unable to identify patients that are most likely to benefit from antibiotics when applying conventional (i.e. univariable or one-variable-at-a-time) subgroup analysis. We updated the systematic review and investigated whether multivariable prediction of patient-level prognosis and antibiotic treatment effect may lead to more tailored treatment assignment in adults presenting to primary care with ARS. METHODS: An IPD-MA of nine double-blind placebo-controlled trials of antibiotic treatment (n=2539) was conducted, with the probability of being cured at 8-15 days as the primary outcome. A logistic mixed effects model was developed to predict the probability of being cured based on demographic characteristics, signs and symptoms, and antibiotic treatment assignment. Predictive performance was quantified based on internal-external cross-validation in terms of calibration and discrimination performance, overall model fit, and the accuracy of individual predictions. RESULTS: Results indicate that the prognosis with respect to risk of cure could not be reliably predicted (c-statistic 0.58 and Brier score 0.24). Similarly, patient-level treatment effect predictions did not reliably distinguish between those that did and did not benefit from antibiotics (c-for-benefit 0.50). CONCLUSIONS: In conclusion, multivariable prediction based on patient demographics and common signs and symptoms did not reliably predict the patient-level probability of cure and antibiotic effect in this IPD-MA. Therefore, these characteristics cannot be expected to reliably distinguish those that do and do not benefit from antibiotics in adults presenting to primary care with ARS.
RESUMEN
Most diagnostic studies exclude missing values and inconclusive results from the analysis or apply simple methods resulting in biased accuracy estimates. This may be due to the lack of availability or awareness of appropriate methods. This scoping review aimed to provide an overview of strategies to handle missing values and inconclusive results in the reference standard or index test in diagnostic accuracy studies. Conducting a systematic literature search in MEDLINE, Cochrane Library, and Web of Science, we could identify many articles proposing methods for addressing missing values in the reference standard. There are also several articles describing methods regarding missing values or inconclusive results in the index test. The latter encompass imputation, frequentist and Bayesian likelihood, model-based, and latent class methods. While methods for missing values in the reference standard are regularly applied in practice, this is not true for methods addressing missing values and inconclusive results in the index test. Our comprehensive overview and description of available methods may raise further awareness of these methods and will enhance their application. Future research is needed to compare the performance of these methods under different conditions to give valid and robust recommendations for their usage in various diagnostic accuracy research scenarios.
Asunto(s)
Diagnóstico , Estándares de Referencia , Teorema de Bayes , Sensibilidad y Especificidad , HumanosRESUMEN
PURPOSE: Although elective surgery is generally safe, some procedures remain associated with an increased risk of complications. Improved preoperative risk stratification and earlier recognition of these complications may ameliorate postoperative recovery and improve long-term outcomes. The perioperative longitudinal study of complications and long-term outcomes (PLUTO) cohort aims to establish a comprehensive biorepository that will facilitate research in this field. In this profile paper, we will discuss its design rationale and opportunities for future studies. PARTICIPANTS: Patients undergoing elective intermediate to high-risk non-cardiac surgery are eligible for enrolment. For the first seven postoperative days, participants are subjected to daily bedside visits by dedicated observers, who adjudicate clinical events and perform non-invasive physiological measurements (including handheld spirometry and single-channel electroencephalography). Blood samples and microbiome specimens are collected at preselected time points. Primary study outcomes are the postoperative occurrence of nosocomial infections, major adverse cardiac events, pulmonary complications, acute kidney injury and delirium/acute encephalopathy. Secondary outcomes include mortality and quality of life, as well as the long-term occurrence of psychopathology, cognitive dysfunction and chronic pain. FINDINGS TO DATE: Enrolment of the first participant occurred early 2020. During the inception phase of the project (first 2 years), 431 patients were eligible of whom 297 patients consented to participate (69%). Observed event rate was 42% overall, with the most frequent complication being infection. FUTURE PLANS: The main purpose of the PLUTO biorepository is to provide a framework for research in the field of perioperative medicine and anaesthesiology, by storing high-quality clinical data and biomaterials for future studies. In addition, PLUTO aims to establish a logistical platform for conducting embedded clinical trials. TRIAL REGISTRATION NUMBER: NCT05331118.
Asunto(s)
Bancos de Muestras Biológicas , Calidad de Vida , Humanos , Diagnóstico Precoz , Estudios Longitudinales , Complicaciones Posoperatorias/diagnóstico , Complicaciones Posoperatorias/epidemiologíaRESUMEN
BACKGROUND: Normal-tissue complication probability (NTCP) models predict complication risk in patients receiving radiotherapy, considering radiation dose to healthy tissues, and are used to select patients for proton therapy, based on their expected reduction in risk after proton therapy versus photon radiotherapy (ΔNTCP). Recommended model evaluation measures include area under the receiver operating characteristic curve (AUC), overall calibration (CITL), and calibration slope (CS), whose precise relation to patient selection is still unclear. We investigated how each measure relates to patient selection outcomes. METHODS: The model validation and consequent patient selection process was simulated within empirical head and neck cancer patient data. By manipulating performance measures independently via model perturbations, the relation between model performance and patient selection was studied. RESULTS: Small reductions in AUC (-0.02) yielded mean changes in ΔNTCP between 0.9-3.2 %, and single-model patient selection differences between 2-19 %. Deviations (-0.2 or +0.2) in CITL or CS yielded mean changes in ΔNTCP between 0.3-1.4 %, and single-model patient selection differences between 1-10 %. CONCLUSIONS: Each measure independently impacts ΔNTCP and patient selection and should thus be assessed in a representative sufficiently large external sample. Our suggested practical model selection approach is considering the model with the highest AUC, and recalibrating it if needed.
Asunto(s)
Neoplasias de Cabeza y Cuello , Terapia de Protones , Humanos , Terapia de Protones/efectos adversos , Selección de Paciente , Dosificación Radioterapéutica , Neoplasias de Cabeza y Cuello/etiología , Probabilidad , Planificación de la Radioterapia Asistida por ComputadorRESUMEN
OBJECTIVES: To assess improvement in the completeness of reporting coronavirus (COVID-19) prediction models after the peer review process. STUDY DESIGN AND SETTING: Studies included in a living systematic review of COVID-19 prediction models, with both preprint and peer-reviewed published versions available, were assessed. The primary outcome was the change in percentage adherence to the transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD) reporting guidelines between pre-print and published manuscripts. RESULTS: Nineteen studies were identified including seven (37%) model development studies, two external validations of existing models (11%), and 10 (53%) papers reporting on both development and external validation of the same model. Median percentage adherence among preprint versions was 33% (min-max: 10 to 68%). The percentage adherence of TRIPOD components increased from preprint to publication in 11/19 studies (58%), with adherence unchanged in the remaining eight studies. The median change in adherence was just 3 percentage points (pp, min-max: 0-14 pp) across all studies. No association was observed between the change in percentage adherence and preprint score, journal impact factor, or time between journal submission and acceptance. CONCLUSIONS: The preprint reporting quality of COVID-19 prediction modeling studies is poor and did not improve much after peer review, suggesting peer review had a trivial effect on the completeness of reporting during the pandemic.
Asunto(s)
COVID-19 , Humanos , COVID-19/epidemiología , Pronóstico , PandemiasRESUMEN
BACKGROUND: Antibiotics are overused in patients with acute rhinosinusitis (ARS) as it is difficult to identify those who benefit from antibiotic treatment. AIM: To develop prediction models for computed tomography (CT)-confirmed ARS and culture-confirmed acute bacterial rhinosinusitis (ABRS) in adults presenting to primary care with symptoms suggestive of ARS. DESIGN AND SETTING: This was a systematic review and individual participant data meta-analysis. METHOD: CT-confirmed ARS was defined as the presence of fluid level or total opacification in any maxillary sinuses, whereas culture-confirmed ABRS was defined by culture of fluid from antral puncture. Prediction models were derived using logistic regression modelling. RESULTS: Among 426 patients from three studies, 140 patients (32.9%) had CT-confirmed ARS. A model consisting of seven variables: previous diagnosis of ARS, preceding upper respiratory tract infection, anosmia, double sickening, purulent nasal discharge on examination, need for antibiotics as judged by a physician, and C-reactive protein (CRP) showed an optimism-corrected c-statistic of 0.73 (95% confidence interval [CI] = 0.69 to 0.78) and a calibration slope of 0.99 (95% CI = 0.72 to 1.19). Among 225 patients from two studies, 68 patients (30.2%) had culture-confirmed ABRS. A model consisting of three variables: pain in teeth, purulent nasal discharge, and CRP showed an optimism-corrected c-statistic of 0.70 (95% CI = 0.63 to 0.77) and a calibration slope of 1.00 (95% CI = 0.66 to 1.52). Clinical utility analysis showed that both models could be useful to rule out the target condition. CONCLUSION: Simple prediction models for CT-confirmed ARS and culture-confirmed ABRS can be useful to safely reduce antibiotic use in adults with ARS in high-prescribing countries.
Asunto(s)
Rinitis , Sinusitis , Enfermedad Aguda , Adulto , Antibacterianos/uso terapéutico , Proteína C-Reactiva , Humanos , Atención Primaria de Salud , Rinitis/diagnóstico por imagen , Rinitis/tratamiento farmacológico , Sinusitis/diagnóstico por imagen , Sinusitis/tratamiento farmacológico , Tomografía Computarizada por Rayos XRESUMEN
BACKGROUND: Clinical prediction models are developed widely across medical disciplines. When predictors in such models are highly collinear, unexpected or spurious predictor-outcome associations may occur, thereby potentially reducing face-validity of the prediction model. Collinearity can be dealt with by exclusion of collinear predictors, but when there is no a priori motivation (besides collinearity) to include or exclude specific predictors, such an approach is arbitrary and possibly inappropriate. METHODS: We compare different methods to address collinearity, including shrinkage, dimensionality reduction, and constrained optimization. The effectiveness of these methods is illustrated via simulations. RESULTS: In the conducted simulations, no effect of collinearity was observed on predictive outcomes (AUC, R2, Intercept, Slope) across methods. However, a negative effect of collinearity on the stability of predictor selection was found, affecting all compared methods, but in particular methods that perform strong predictor selection (e.g., Lasso). Methods for which the included set of predictors remained most stable under increased collinearity were Ridge, PCLR, LAELR, and Dropout. CONCLUSIONS: Based on the results, we would recommend refraining from data-driven predictor selection approaches in the presence of high collinearity, because of the increased instability of predictor selection, even in relatively high events-per-variable settings. The selection of certain predictors over others may disproportionally give the impression that included predictors have a stronger association with the outcome than excluded predictors.
RESUMEN
While the opportunities of ML and AI in healthcare are promising, the growth of complex data-driven prediction models requires careful quality and applicability assessment before they are applied and disseminated in daily practice. This scoping review aimed to identify actionable guidance for those closely involved in AI-based prediction model (AIPM) development, evaluation and implementation including software engineers, data scientists, and healthcare professionals and to identify potential gaps in this guidance. We performed a scoping review of the relevant literature providing guidance or quality criteria regarding the development, evaluation, and implementation of AIPMs using a comprehensive multi-stage screening strategy. PubMed, Web of Science, and the ACM Digital Library were searched, and AI experts were consulted. Topics were extracted from the identified literature and summarized across the six phases at the core of this review: (1) data preparation, (2) AIPM development, (3) AIPM validation, (4) software development, (5) AIPM impact assessment, and (6) AIPM implementation into daily healthcare practice. From 2683 unique hits, 72 relevant guidance documents were identified. Substantial guidance was found for data preparation, AIPM development and AIPM validation (phases 1-3), while later phases clearly have received less attention (software development, impact assessment and implementation) in the scientific literature. The six phases of the AIPM development, evaluation and implementation cycle provide a framework for responsible introduction of AI-based prediction models in healthcare. Additional domain and technology specific research may be necessary and more practical experience with implementing AIPMs is needed to support further guidance.
RESUMEN
OBJECTIVE: To provide approximations to recover the full regression equation across different scenarios of incompletely reported prediction models that were developed from binary logistic regression. STUDY DESIGN AND SETTING: In a case study, we considered four common scenarios and illustrated their corresponding approximations: (A) Missing: the intercept, Available: the regression coefficients of predictors, overall frequency of the outcome and descriptive statistics of the predictors; (B) Missing: regression coefficients and the intercept, Available: a simplified score; (C) Missing: regression coefficients and the intercept, Available: a nomogram; (D) Missing: regression coefficients and the intercept, Available: a web calculator. RESULTS: In the scenario A, a simplified approach based on the predicted probability corresponding to the average linear predictor was inaccurate. An approximation based on the overall outcome frequency and an approximation of the linear predictor distribution was more accurate, however, the appropriateness of the underlying assumptions cannot be verified in practice. In the scenario B, the recovered equation was inaccurate due to rounding and categorization of risk scores. In the scenarios C and D, the full regression equation could be recovered with minimal error. CONCLUSION: The accuracy of the approximations in recovering the regression equation varied depending on the available information.
Asunto(s)
Modelos Logísticos , Recolección de Datos , Humanos , Factores de RiesgoRESUMEN
BACKGROUND: To ensure availability of hospital beds and improve COVID-19 patients' well-being during the ongoing pandemic, hospital care could be offered at home. Retrospective studies show promising results of deploying remote hospital care to reduce the number of days spent in the hospital, but the beneficial effect has yet to be established. METHODS: We conducted a single centre, randomised trial from January to June 2021, including hospitalised COVID-19 patients who were in the recovery stage of the disease. Hospital care for the intervention group was transitioned to the patient's home, including oxygen therapy, medication and remote monitoring. The control group received in-hospital care as usual. The primary endpoint was the number of hospital-free days during the 30 days following randomisation. Secondary endpoints included health care consumption during the follow-up period and mortality. RESULTS: A total of 62 patients were randomised (31 control, 31 intervention). The mean difference in hospital-free days was 1.7 (26.7 control vs. 28.4 intervention, 95% CI of difference -0.5 to 4.2, p = 0.112). In the intervention group, the index hospital length of stay was 1.6 days shorter (95% CI -2.4 to -0.8, p < 0.001), but the total duration of care under hospital responsibility was 4.1 days longer (95% CI 0.5 to 7.7, p = 0.028). CONCLUSION: Remote hospital care for recovering COVID-19 patients is feasible. However, we could not demonstrate an increase in hospital-free days in the 30 days following randomisation. Optimising the intervention, timing, and identification of patients who will benefit most from remote hospital care could improve the impact of this intervention.