Methodological guidance for the evaluation and updating of clinical prediction models: a systematic review.

BACKGROUND: Clinical prediction models are often not evaluated properly in specific settings or updated, for instance, with information from new markers. These key steps are needed such that models are fit for purpose and remain relevant in the long-term. We aimed to present an overview of methodological guidance for the evaluation (i.e., validation and impact assessment) and updating of clinical prediction models. METHODS: We systematically searched nine databases from January 2000 to January 2022 for articles in English with methodological recommendations for the post-derivation stages of interest. Qualitative analysis was used to summarize the 70 selected guidance papers. RESULTS: Key aspects for validation are the assessment of statistical performance using measures for discrimination (e.g., C-statistic) and calibration (e.g., calibration-in-the-large and calibration slope). For assessing impact or usefulness in clinical decision-making, recent papers advise using decision-analytic measures (e.g., the Net Benefit) over simplistic classification measures that ignore clinical consequences (e.g., accuracy, overall Net Reclassification Index). Commonly recommended methods for model updating are recalibration (i.e., adjustment of intercept or baseline hazard and/or slope), revision (i.e., re-estimation of individual predictor effects), and extension (i.e., addition of new markers). Additional methodological guidance is needed for newer types of updating (e.g., meta-model and dynamic updating) and machine learning-based models. CONCLUSION: Substantial guidance was found for model evaluation and more conventional updating of regression-based models. An important development in model evaluation is the introduction of a decision-analytic framework for assessing clinical usefulness. Consensus is emerging on methods for model updating.

Developing a prediction model for the self-evaluation of erectile dysfunction in an adult male population.

The purpose of this study was to develop an erectile dysfunction (ED) risk assessment tool suitable for the general population. Based on an ED network survey of the general adult male population in China from October to November 2019, a total of 612 cases with a mean age of 31 years (interquartile range: 28-37) with valid data were collected: 357 cases were assigned to the training set and 255 to the validation set. The ED risk prediction model was established by multifactor logistic regression analysis, and nomograms were constructed for visualisation. In the validation set, a receiver operating characteristic curve, calibration curve analysis and decision curve analysis were used to evaluate the discrimination, calibration and clinical usefulness of the ED risk prediction model. Based on multivariate logistic regression, education, smoking, chronic diseases, feelings about one's spouse, frequency of sexual intercourse, masturbation and self-reported sexual satisfaction were selected as predictors to develop the ED prediction model. The model had good discrimination, calibration and clinical applicability. The ED risk prediction model developed in this study can effectively predict ED risk in the general population.

The Impact of Rapid Exome Sequencing on Medical Management of Critically Ill Children.

OBJECTIVES: To evaluate the clinical usefulness of rapid exome sequencing (rES) in critically ill children with likely genetic disease using a standardized process at a single institution. To provide evidence that rES with should become standard of care for this patient population. STUDY DESIGN: We implemented a process to provide clinical-grade rES to eligible children at a single institution. Eligibility included (a) recommendation of rES by a consulting geneticist, (b) monogenic disorder suspected, (c) rapid diagnosis predicted to affect inpatient management, (d) pretest counseling provided by an appropriate provider, and (e) unanimous approval by a committee of 4 geneticists. Trio exome sequencing was sent to a reference laboratory that provided verbal report within 7-10 days. Clinical outcomes related to rES were prospectively collected. Input from geneticists, genetic counselors, pathologists, neonatologists, and critical care pediatricians was collected to identify changes in management related to rES. RESULTS: There were 54 patients who were eligible for rES over a 34-month study period. Of these patients, 46 underwent rES, 24 of whom (52%) had at least 1 change in management related to rES. In 20 patients (43%), a molecular diagnosis was achieved, demonstrating that nondiagnostic exomes could change medical management in some cases. Overall, 84% of patients were under 1 month old at rES request and the mean turnaround time was 9 days. CONCLUSIONS: rES testing has a significant impact on the management of critically ill children with suspected monogenic disease and should be considered standard of care for tertiary institutions who can provide coordinated genetics expertise.

Semi-supervised learning to improve generalizability of risk prediction models.

The utility of a prediction model depends on its generalizability to patients drawn from different but related populations. We explored whether a semi-supervised learning model could improve the generalizability of colorectal cancer (CRC) risk prediction relative to supervised learning methods. Data on 113,141 patients diagnosed with nonmetastatic CRC from 2004 to 2012 were obtained from the Surveillance Epidemiology End Results registry for model development, and data on 1149 patients from the Second Affiliated Hospital, Zhejiang University School of Medicine, who were diagnosed between 2004 and 2011, were collected for generalizability testing. A clinical prediction model for CRC survival risk using a semi-supervised logistic regression method was developed and validated to investigate the model discrimination, calibration, generalizability, interpretability and clinical usefulness. Rigorous model performance comparisons with other supervised learning models were performed. The area under the curve of the logistic membership model revealed a large heterogeneity between the development cohort and validation cohort, which is typical of generalizability studies of prediction models. The discrimination was good for all models. Calibration was poor for supervised learning models, while the semi-supervised logistic regression model exhibited a good calibration on the validation cohort, which indicated good generalizability. Clinical usefulness analysis showed that semi-supervised logistic regression can lead to better clinical outcomes than supervised learning methods. These results increase our current understanding of the generalizability of different models and provide a reference for predictive model development for clinical decision-making.

External Validation and Clinical Usefulness of First Trimester Prediction Models for the Risk of Preeclampsia: A Prospective Cohort Study.

INTRODUCTION: This study assessed the external validity of all published first trimester prediction models for the risk of preeclampsia (PE) based on routinely collected maternal predictors. Moreover, the potential utility of the best-performing models in clinical practice was evaluated. MATERIAL AND METHODS: Ten prediction models were systematically selected from the literature. We performed a multicenter prospective cohort study in the Netherlands between July 1, 2013, and December 31, 2015. Eligible pregnant women completed a web-based questionnaire before 16 weeks' gestation. The outcome PE was established using postpartum questionnaires and medical records. Predictive performance of each model was assessed by means of discrimination (c-statistic) and a calibration plot. Clinical usefulness was evaluated by means of decision curve analysis and by calculating the potential impact at different risk thresholds. RESULTS: The validation cohort contained 2,614 women of whom 76 developed PE (2.9%). Five models showed moderate discriminative performance with c-statistics ranging from 0.73 to 0.77. Adequate calibration was obtained after refitting. The best models were clinically useful over a small range of predicted probabilities. DISCUSSION: Five of the ten included first trimester prediction models for PE showed moderate predictive performance. The best models may provide more benefit compared to risk selection as used in current guidelines.

Clinical Usefulness of 18F-Fluorodeoxyglucose Positron Emission Tomography in the Diagnostic Algorithm of Advanced Entero-Pancreatic Neuroendocrine Neoplasms.

BACKGROUND: The role of 18F-fluorodeoxyglucose positron emission tomography (18F-FDG PET) in the diagnostic algorithm of entero-pancreatic neuroendocrine neoplasms (EP NENs) is unclear because most available data derive from heterogeneous populations in terms of tumor biology and disease status at time of examination. The aim of this study was to determine the ability of 18F-FDG PET to identify patients with more aggressive disease among those with advanced EP NENs. Subjects, Materials, and Methods . Patients with advanced EP NENs and known disease status (progressive disease [PD] or stable disease [SD]) according to imaging procedures, who received 18F-FDG PET and computed tomography scans during a time frame of 1 month, were included. RESULTS: A total of 93 patients, including 69 patients with pancreatic NENs and 24 patients with small-intestine NENs, were included. At the time of study entry, 64 patients (68.8%) had PD, and the remaining 29 patients (31.2%) had SD. A total of 62 patients (66.7%) had positive 18F-FDG PET, whereas 18F-FDG PET was negative in the remaining 31 patients (33.3%). Overall, 18F-FDG PET sensitivity and specificity to detect PD were 90.6% and 86.2%, respectively, resulting in a diagnostic accuracy of 89.2%. A positive 18F-FDG PET was significantly associated with PD at the time of study entry (p < .0001 at multivariate analysis). Although a higher proportion of 18F-FDG PET-positive examinations were observed in patients with higher tumor grade (p = .01), 53.8% of patients with grade 1 neuroendocrine tumors (NETs) had positive 18F-FDG PET, and 37.5% of patients with grade 2 NETs had negative 18F-FDG PET. Overall survival was significantly shorter in 18F-FDG PET-positive patients (median: 60 months) in comparison with 18F-FDG PET-negative patients (median not reached; p = .008). CONCLUSION: 18F-FDG PET has a high diagnostic accuracy to identify progression of disease with unfavorable clinical outcome in patients with advanced EP NENs. Knowledge of disease status and G grading are key factors for physicians to better select patients for whom 18F-FDG PET is clinically useful. IMPLICATIONS FOR PRACTICE: The findings of the present study may help physicians dealing with advanced neuroendocrine neoplasms to select patients for whom 18F-fluorodeoxyglucose positron emission tomography is useful to predict poor clinical outcome.

The frontal assessment battery in clinical practice: a systematic review.

BACKGROUND: The frontal assessment battery (FAB) is a brief tool designed to evaluate executive function. Some studies have particularly focused on assessing its applicability addressing two issues: first, on detecting the brain regions responsible for the FAB performance, and second, on determining its capability for differential diagnosis. Our aim was to summarize and analyze critically the studies that assessed the neuroanatomical correspondence and the differential diagnostic value of the FAB in several study populations suffering from different pathologies. METHODS: We completed a literature search in MEDLINE (via PubMed) database by using the term "frontal assessment battery" and the combination of this term with "applicability" or "use" or "usefulness". The search was limited to articles in English or Spanish languages, published between 1 September 2000 and 30 September 2016, human studies, and journal articles. RESULTS: A total of 32 studies met inclusion criteria. Seventeen studies were aimed at identifying the brain regions or the neural substrates involved in executive functions measured by the FAB and 15 studies at verifying that the FAB was an appropriate tool for the differential diagnosis in neurological diseases. CONCLUSION: Our study showed that the FAB may be an adequate assessment tool for executive function and may provide useful information for differential diagnosis in several diseases. Given that the FAB takes short time and is easy to administer, its usage may be of great interest as part of a full neuropsychological assessment in clinical settings. Copyright © 2017 John Wiley & Sons, Ltd.

Beyond discrimination: A comparison of calibration methods and clinical usefulness of predictive models of readmission risk.

BACKGROUND: Prior to implementing predictive models in novel settings, analyses of calibration and clinical usefulness remain as important as discrimination, but they are not frequently discussed. Calibration is a model's reflection of actual outcome prevalence in its predictions. Clinical usefulness refers to the utilities, costs, and harms of using a predictive model in practice. A decision analytic approach to calibrating and selecting an optimal intervention threshold may help maximize the impact of readmission risk and other preventive interventions. OBJECTIVES: To select a pragmatic means of calibrating predictive models that requires a minimum amount of validation data and that performs well in practice. To evaluate the impact of miscalibration on utility and cost via clinical usefulness analyses. MATERIALS AND METHODS: Observational, retrospective cohort study with electronic health record data from 120,000 inpatient admissions at an urban, academic center in Manhattan. The primary outcome was thirty-day readmission for three causes: all-cause, congestive heart failure, and chronic coronary atherosclerotic disease. Predictive modeling was performed via L1-regularized logistic regression. Calibration methods were compared including Platt Scaling, Logistic Calibration, and Prevalence Adjustment. Performance of predictive modeling and calibration was assessed via discrimination (c-statistic), calibration (Spiegelhalter Z-statistic, Root Mean Square Error [RMSE] of binned predictions, Sanders and Murphy Resolutions of the Brier Score, Calibration Slope and Intercept), and clinical usefulness (utility terms represented as costs). The amount of validation data necessary to apply each calibration algorithm was also assessed. RESULTS: C-statistics by diagnosis ranged from 0.7 for all-cause readmission to 0.86 (0.78-0.93) for congestive heart failure. Logistic Calibration and Platt Scaling performed best and this difference required analyzing multiple metrics of calibration simultaneously, in particular Calibration Slopes and Intercepts. Clinical usefulness analyses provided optimal risk thresholds, which varied by reason for readmission, outcome prevalence, and calibration algorithm. Utility analyses also suggested maximum tolerable intervention costs, e.g., $1720 for all-cause readmissions based on a published cost of readmission of $11,862. CONCLUSIONS: Choice of calibration method depends on availability of validation data and on performance. Improperly calibrated models may contribute to higher costs of intervention as measured via clinical usefulness. Decision-makers must understand underlying utilities or costs inherent in the use-case at hand to assess usefulness and will obtain the optimal risk threshold to trigger intervention with intervention cost limits as a result.

Towards better clinical prediction models: seven steps for development and an ABCD for validation.

Clinical prediction models provide risk estimates for the presence of disease (diagnosis) or an event in the future course of disease (prognosis) for individual patients. Although publications that present and evaluate such models are becoming more frequent, the methodology is often suboptimal. We propose that seven steps should be considered in developing prediction models: (i) consideration of the research question and initial data inspection; (ii) coding of predictors; (iii) model specification; (iv) model estimation; (v) evaluation of model performance; (vi) internal validation; and (vii) model presentation. The validity of a prediction model is ideally assessed in fully independent data, where we propose four key measures to evaluate model performance: calibration-in-the-large, or the model intercept (A); calibration slope (B); discrimination, with a concordance statistic (C); and clinical usefulness, with decision-curve analysis (D). As an application, we develop and validate prediction models for 30-day mortality in patients with an acute myocardial infarction. This illustrates the usefulness of the proposed framework to strengthen the methodological rigour and quality for prediction models in cardiovascular research.

An assessment of the usefulness of Kinesiograph as an aid in the diagnosis of TMD: a review of Manfredini et al.'s studies.

AIM: Performing a literature review of publications by Dr. Manfredini et al. related to their temporomandibular joint (TMJ) injection therapy outcome with conclusions on the clinical utility of computerized measurement devices used in the management of temporomandibular disorders (TMDs). In addition, reviewing their published opinion on an occlusion: TMD versus a biopsychosocial paradigm for TMD. Manfredini et al. authored an article published in the Journal of the American Dental Association (JADA) 2013, "An Assessment of the usefulness of jaw kinesiography in monitoring temporomandibular disorders," the most recent of 12 articles. In all studies, subjects received TMJ injections with an objective measurement outcome criterion; increased maximum mouth opening (MMO) and subjective symptom improvement of pain and chewing function. In the 2013 JADA article, the Mandibular Kinesiograph, referred to as KG, measured MMO before and after therapy. In 11 prior articles, all subject groups with limited mouth opening exhibited very significant increased MMO post-treatment, documenting treatment success using the same 2013 protocol. The 2013 study showed a 1·1 mm improved MMO, described as insignificant. The authors did not critique or explain the aberrant, skewed 2013 outcome data contrasted with their prior studies, which showed overwhelmingly significant increased MMO. Instead, they concluded that the MMO recording device was clinically useless. This motivated a literature review of the authors' TMD publications. CONCLUSION: The publications by Manfredini et al. recognized proponents of the psychosocial model of TMD, including the 2013 article, appear to be part of a campaign denying an occlusion: TMD relationship and disparaging the specific computerized measurement devices and the dentists using them in the management of their TMD patients using neuromuscular occlusion dental treatment.

Exploring the Clinical Usefulness of Undergraduate Medical Research: A Mixed-Methods Study.

A significant amount of published clinical research has no measurable impact on health and disease outcomes, and research in undergraduate medical education is viewed as especially susceptible. The aims of this mixed methods study were to (a) to use group concept mapping (GCM) to explore key features identified by hospital physicians, medical educators, and medical students as central to clinical usefulness in an undergraduate medical research context, and (b) review a sample of undergraduate medical research projects based on usefulness criteria described by Ioannidis (2016). In the GCM procedure, 54 respondents (39 students, 15 physicians) from an Irish medical school participated across each of three phases: brainstorming, sorting, and rating. Data was analysed using multidimensional scaling and hierarchical clustering. A retrospective analysis of 252 student projects was also completed using a rubric based on Ioannidis's (2016) six domains of "clinical usefulness": problem base, context placement and information gain, pragmatism, patient-centredness, feasibility, and transparency. Projects were scored for each domain by three assessors. Results were analysed and presented using descriptive analysis.GCM analysis revealed the following "clinically useful" research characteristics: optimal design and methodology, practicality, research skills development, translational impact, patient-centredness, and asking a clinical question. Following a rubric-based analysis of projects, the highest scoring categories (mean rating; range of 1-4) were feasibility (3.57), transparency (3.32), and problem base (3.05). The lowest scoring areas were context placement and information gain (2.73), pragmatism (2.68), and patient-centredness (212). We identified considerable conceptual overlap between stakeholder consensus views on "clinical usefulness" as applied to undergraduate research and Ioannidis's criteria. Patient-centredness was identified as a domain requiring greater emphasis during the design of undergraduate medical research. Supplementary Information: The online version contains supplementary material available at 10.1007/s40670-024-02035-7.

Predictors of seizure detection and EEG clinical impact in an italian tertiary emergency department.

BACKGROUND AND OBJECTIVES: Epileptic seizures pose challenges in emergency departments (ED), affecting up to 10% of admitted patients. This study aimed to assess emergency electroencephalogram (EmEEG) utilization, identifying factors predicting seizure detection and its influence on clinical decisions. METHODS: A retrospective review of 1135 EmEEGs on 1017 patients at a tertiary teaching hospital between June 2022 and June 2023 was conducted. Data included demographics, medical history, EmEEG indications, neuroimaging findings, and clinical outcomes. Statistical analyses utilized Fisher's exact tests and logistic regression models. RESULTS: EmEEG detected status epilepticus-related seizures in 5.40% of cases, seizures without status epilepticus in 3.05%, and status epilepticus without discrete seizures in 3.74%. Epileptiform abnormalities were noted in 22.12% of EmEEGs. EmEEG influenced initial diagnoses (21.24%), antiseizure medication changes (20.85%), and discharge decisions (39.04%). Predictors for seizures/status epilepticus included previous neurosurgery, seizures in the ED, and cognitive/behavioral impairment (p < 0.001). EmEEG significantly altered initial diagnoses based on witnessed seizures, involuntary movements, epileptiform abnormalities, and 1-2 Hz generalized periodic discharges (p < 0.001). Changes in antiseizure medications correlated with seizure occurrence, neuroimaging results, epileptiform abnormalities, and EEG background slowing (p < 0.001). Factors influencing discharge decisions included previous neurosurgery, consciousness impairment, acute neuroimaging pathology, EEG focal slowing, and EEG background slowing (p < 0.001). DISCUSSION: The study clarifies EmEEG's role in modifying initial diagnoses, treatment approaches, and discharge decisions. The study provides insights into the nuanced impact of EmEEG in different clinical scenarios, offering valuable guidance for clinicians in selecting patients for EmEEG, particularly in conditions of limited EEG availability.

Decision Curve Analysis for Personalized Treatment Choice between Multiple Options.

BACKGROUND: Decision curve analysis can be used to determine whether a personalized model for treatment benefit would lead to better clinical decisions. Decision curve analysis methods have been described to estimate treatment benefit using data from a single randomized controlled trial. OBJECTIVES: Our main objective is to extend the decision curve analysis methodology to the scenario in which several treatment options exist and evidence about their effects comes from a set of trials, synthesized using network meta-analysis (NMA). METHODS: We describe the steps needed to estimate the net benefit of a prediction model using evidence from studies synthesized in an NMA. We show how to compare personalized versus one-size-fit-all treatment decision-making strategies, such as "treat none" or "treat all patients with a specific treatment" strategies. First, threshold values for each included treatment need to be defined (i.e., the minimum risk difference compared with control that renders a treatment worth taking). The net benefit per strategy can then be plotted for a plausible range of threshold values to reveal the most clinically useful strategy. We applied our methodology to an NMA prediction model for relapsing-remitting multiple sclerosis, which can be used to choose between natalizumab, dimethyl fumarate, glatiramer acetate, and placebo. RESULTS: We illustrated the extended decision curve analysis methodology using several threshold value combinations for each available treatment. For the examined threshold values, the "treat patients according to the prediction model" strategy performs either better than or close to the one-size-fit-all treatment strategies. However, even small differences may be important in clinical decision making. As the advantage of the personalized model was not consistent across all thresholds, improving the existing model (by including, for example, predictors that will increase discrimination) is needed before advocating its clinical usefulness. CONCLUSIONS: This novel extension of decision curve analysis can be applied to NMA-based prediction models to evaluate their use to aid treatment decision making. HIGHLIGHTS: Decision curve analysis is extended into a (network) meta-analysis framework.Personalized models predicting treatment benefit are evaluated when several treatment options are available and evidence about their effects comes from a set of trials.Detailed steps to compare personalized versus one-size-fit-all treatment decision-making strategies are outlined.This extension of decision curve analysis can be applied to (network) meta-analysis-based prediction models to evaluate their use to aid treatment decision making.

External Validation of Finnish Diabetes Risk Score and Australian Diabetes Risk Assessment Tool Prediction Models to Identify People with Undiagnosed Type 2 Diabetes: A Cross-sectional Study in Iran.

Background: Noninvasive risk prediction models have been widely used in various settings to identify individuals with undiagnosed diabetes. Objectives: We aimed to evaluate the discrimination, calibration, and clinical usefulness of the Finnish Diabetes Risk Score (FINDRISC) and Australian Diabetes Risk Assessment (AUSDRISK) to screen undiagnosed diabetes in Kerman, Iran. Methods: We analyzed data from 2014 to 2018 in the second round of the Kerman Coronary Artery Disease Risk Factors Study (KERCADRS), Iran. Participants aged 35 - 65 with no history of confirmed diabetes were eligible. The area under the receiver operating characteristic curve (AUROC) and decision curve analysis were applied to evaluate the discrimination power and clinical usefulness of the models, respectively. The calibration was assessed by the Hosmer-Lemeshow test and the calibration plots. Results: Out of 3262 participants, 145 (4.44%) had undiagnosed diabetes. The estimated AUROCs were 0.67 and 0.62 for the AUSDRISK and FINDRISC models, respectively (P < 0.001). The chi-square test results for FINDRISC and AUSDRISC were 7.90 and 16.47 for the original model and 3.69 and 14.61 for the recalibrated model, respectively. Based on the decision curves, useful threshold ranges for the original models of FINDRIS and AUSDRISK were 4% to 10% and 3% to 13%, respectively. Useful thresholds for the recalibrated models of FINDRISC and AUSDRISK were 4% to 8% and 4% to 9%, respectively. Conclusions: The original AUSDRISK model performs better than FINDRISC in identifying patients with undiagnosed diabetes and could be used as a simple and noninvasive tool where access to laboratory facilities is costly or limited.

Utility of Antibodies in the Diagnoses of Thyroid Diseases: A Review Article.

Thyroid problems are among the most widespread endocrine illnesses, affecting individuals in India and the global population. A thyroid function test is used to diagnose, screen, and monitor patients. Hyperthyroidism is a clinical condition due to excessive circulation of thyroid hormone; in contrast, hypothyroidism is due to a deficiency of thyroid hormone. Graves' disease (GD) is a form of hyperthyroidism due to thyroid-stimulating hormone receptor autoantibodies (TRAb), and anti-thyroid peroxidase antibodies (anti-TPO antibodies). The most common reason for hypothyroidism is Hashimoto's thyroiditis (HT), in which patients have thyroid receptor antibodies (TRAb), antibodies to thyroid peroxidase (TPO), and thyroglobulin antibodies. Many essential genes, including the thyroid-specific genes thyroglobulin (TSGT), TSH-receptor gene, human leukocyte antigen (HLA) genes, cytotoxic T lymphocyte-associated antigen (CTLA) genes, thyroglobulin gene, vitamin D receptor gene, and many immune-regulatory genes were associated with autoimmune thyroid diseases' (AITDs') etiology. This review paper aims to determine if antibodies are beneficial in detecting autoimmune thyroid disease or not. We have also discussed the etiology of autoimmune thyroid illness, serum antibodies in autoimmune thyroid disease, pathophysiology, and TSH receptor features.

The usefulness of the Electronic Patient Visit Assessment (ePVA)© as a clinical support tool for real-time interventions in head and neck cancer.

BACKGROUND: Patients with head and neck cancer (HNC) experience painful, debilitating symptoms and functional limitations that can interrupt cancer treatment, and decrease their health-related quality of life (HRQoL). The Electronic Patient Visit Assessment (ePVA) for head and neck is a web-based mHealth patient-reported measure that asks questions about 21 categories of symptoms and functional limitations common to HNC. This article presents the development and usefulness of the ePVA as a clinical support tool for real-time interventions for patient-reported symptoms and functional limitations in HNC. METHODS: Between January 2018 and August 2019, 75 participants were enrolled in a clinical usefulness study of the ePVA. Upon signing informed consent, participants completed the ePVA and the European Organization for Research and Treatment of Cancer (EORTC) Quality of Life Questionnaire (QLQ) general (C30) questionnaire v3.0 (scores range from 0 to 100 with 100 representing best HRQoL). Clinical usefulness of the ePVA was defined as demonstration of reliability, convergent validity with HRQoL, and acceptability of the ePVA (i.e., >70% of eligible participants complete the ePVA at two or more visits and >70% of ePVA reports are read by providers). Formal focus group discussions with the interdisciplinary team that cared for patients with HNC guided the development of the ePVA as a clinical support tool. Qualitative and quantitative methods were used throughout the study. Descriptive statistics consisting of means and frequencies, Pearson correlation coefficient, and Student's t-tests were calculated using SAS 9.4 and STATA. RESULTS: The participants were primarily male (71%), White (76%), diagnosed with oropharyngeal or oral cavity cancers (53%), and undergoing treatment for HNC (69%). Data analyses supported the reliability (alpha =0.85), convergent validity with HRQoL scores, and acceptability of the ePVA. Participants with the highest number of symptoms and functional limitations reported significantly worse HRQoL (sum of symptoms: r=-0.50, P<0.0001; sum of function limitations: r=-0.56, P<0.0001). Ninety-two percent of participants (59 of 64) who had follow-up visits within the 6-month study period completed the ePVA at two or more visits and providers read 89% (169 of 189) of automated ePVA reports. The use of the ePVA as a clinical support tool for real-time interventions for symptoms and functional limitations reported by patients is described in a clinical exemplar. CONCLUSIONS: This research indicates that the ePVA may be a useful mHealth tool as a clinical support tool for real-time interventions for patient-reported symptoms and functional limitations in HNC. The study findings support future translational research to enhance the usefulness of the ePVA in real world settings for early interventions that decrease symptom burden and improve the QoL of patients with HNC.

A new nomogram for the prediction of bone metastasis in patients with prostate cancer.

OBJECTIVE: This study aimed to establish a new prognostic nomogram for bone metastasis in patients with prostate cancer (PCa). METHODS: This study retrospectively analyzed clinical data from 332 patients diagnosed with PCa from 2014 to 2019, and patients were randomly divided into a training set (n = 184) and a validation set (n = 148). Multivariate logistic regression analysis was used to establish a prediction model based on the training set, and a nomogram was constructed for visual presentation. The calibration, discrimination and clinical usefulness of the model were evaluated using the validation set. RESULTS: Total prostate-specific antigen, clinical tumor stage, Gleason score, prostate volume, red cell distribution width and serum alkaline phosphatase were selected as predictors to develop a prediction model of bone metastasis. After evaluation, the model developed in our study exhibited good discrimination (area under the curve: 0.958; 95% confidence interval: 0.93-0.98), calibration (U = 0.01) and clinical usefulness. CONCLUSIONS: The new proposed model showed high accuracy for bone metastasis prediction in patients with PCa and good clinical usefulness.

Development, validation and clinical usefulness of a prognostic model for relapse in relapsing-remitting multiple sclerosis.

BACKGROUND: Prognosis for the occurrence of relapses in individuals with relapsing-remitting multiple sclerosis (RRMS), the most common subtype of multiple sclerosis (MS), could support individualized decisions and disease management and could be helpful for efficiently selecting patients for future randomized clinical trials. There are only three previously published prognostic models on this, all of them with important methodological shortcomings. OBJECTIVES: We aim to present the development, internal validation, and evaluation of the potential clinical benefit of a prognostic model for relapses for individuals with RRMS using real-world data. METHODS: We followed seven steps to develop and validate the prognostic model: (1) selection of prognostic factors via a review of the literature, (2) development of a generalized linear mixed-effects model in a Bayesian framework, (3) examination of sample size efficiency, (4) shrinkage of the coefficients, (5) dealing with missing data using multiple imputations, (6) internal validation of the model. Finally, we evaluated the potential clinical benefit of the developed prognostic model using decision curve analysis. For the development and the validation of our prognostic model, we followed the TRIPOD statement. RESULTS: We selected eight baseline prognostic factors: age, sex, prior MS treatment, months since last relapse, disease duration, number of prior relapses, expanded disability status scale (EDSS) score, and number of gadolinium-enhanced lesions. We also developed a web application that calculates an individual's probability of relapsing within the next 2 years. The optimism-corrected c-statistic is 0.65 and the optimism-corrected calibration slope is 0.92. For threshold probabilities between 15 and 30%, the "treat based on the prognostic model" strategy leads to the highest net benefit and hence is considered the most clinically useful strategy. CONCLUSIONS: The prognostic model we developed offers several advantages in comparison to previously published prognostic models on RRMS. Importantly, we assessed the potential clinical benefit to better quantify the clinical impact of the model. Our web application, once externally validated in the future, could be used by patients and doctors to calculate the individualized probability of relapsing within 2 years and to inform the management of their disease.

Dynamic prediction models improved the risk classification of type 2 diabetes compared with classical static models.

OBJECTIVE: Dynamic prediction models use the repeated measurements of predictors to estimate coefficients that link the longitudinal predictors to a static model (i.e. Cox regression). This study aims to develop and validate a dynamic prediction for incident type 2 diabetes (T2DM) as the outcome. STUDY DESIGN AND SETTING: Data from the Tehran lipid and glucose study was used to develop (n = 5291 individuals; phases 1 to 3) and validate (n = 3147 individuals; phases 3 to 6) the dynamic prediction model among individuals aged ≥ 20 years. We used repeated measurements of fasting plasma glucose (FPG) or waist circumference (WC) in the framework of the joint modeling (JM) of longitudinal and time-to-event analysis. RESULTS: Compared with the Cox which used just baseline data, JM showed the same discrimination, better calibration, and higher clinical usefulness (i.e. with a net benefit considering both true and false positive decisions); all were shown with repeated measurements of FPG/WC. Additionally, in our study, the dynamic models improve the risk reclassification (net reclassification index 33% for FPG and 24% for WC model). CONCLUSION: Dynamic prediction models, compared with the static one could yield significant improvements in the prediction of T2DM. The complexity of the dynamic models could be addressed by using decision support systems.

Clinical Utility of a Nomogram for Predicting 30-Days Poor Outcome in Hospitalized Patients With COVID-19: Multicenter External Validation and Decision Curve Analysis.

Aim: Early detection of coronavirus disease 2019 (COVID-19) patients who are likely to develop worse outcomes is of great importance, which may help select patients at risk of rapid deterioration who should require high-level monitoring and more aggressive treatment. We aimed to develop and validate a nomogram for predicting 30-days poor outcome of patients with COVID-19. Methods: The prediction model was developed in a primary cohort consisting of 233 patients with laboratory-confirmed COVID-19, and data were collected from January 3 to March 20, 2020. We identified and integrated significant prognostic factors for 30-days poor outcome to construct a nomogram. The model was subjected to internal validation and to external validation with two separate cohorts of 110 and 118 cases, respectively. The performance of the nomogram was assessed with respect to its predictive accuracy, discriminative ability, and clinical usefulness. Results: In the primary cohort, the mean age of patients was 55.4 years and 129 (55.4%) were male. Prognostic factors contained in the clinical nomogram were age, lactic dehydrogenase, aspartate aminotransferase, prothrombin time, serum creatinine, serum sodium, fasting blood glucose, and D-dimer. The model was externally validated in two cohorts achieving an AUC of 0.946 and 0.878, sensitivity of 100 and 79%, and specificity of 76.5 and 83.8%, respectively. Although adding CT score to the clinical nomogram (clinical-CT nomogram) did not yield better predictive performance, decision curve analysis showed that the clinical-CT nomogram provided better clinical utility than the clinical nomogram. Conclusions: We established and validated a nomogram that can provide an individual prediction of 30-days poor outcome for COVID-19 patients. This practical prognostic model may help clinicians in decision making and reduce mortality.