Single-center external validation and reconstruction of multiple predictive models for skip lateral lymph node metastasis in papillary thyroid carcinoma.

Objective: The unique metastatic pattern of skip lateral lymph node metastasis (SLLNM) in PTC patients may lead to missed diagnosis of lateral cervical metastatic lymph nodes. Therefore, many different SLLNM prediction models were constructed. In this study, partially eligible models (Hu 2020, Wang 2020, and Zhao 2023 nomograms) were selected for external validation, and then new variables were incorporated for model reconstruction to extend clinical applicability. Methods: 576 PTC patients from our center were selected to evaluate the performance of the three nomograms using the receiver operating characteristic curve (ROC), calibration curves, and decision curve analyses (DCA). Three new variables were added to calibrate the model, including assessment of LN status on ultrasound (US-SLLNM), the distance from the tumor to the capsule (Capsular distance), and the number of central lymph node dissections (CLND number). Univariate and multivariate logistic regression analyses were used to screen independent predictors to reconstruct the model, and 1000 Bootstrap internal validations were performed. Results: SLLNM were present in 69/576 patients (12.0%). In external validation, the area under the ROC curves (AUCs) for Hu 2020, Wang 2020, and Zhao 2023 nomograms were 0.695 (95% CI:0.633-0.766), 0.792 (95% CI=0.73-0.845), and 0.769 (95% CI:0.713-0.824), respectively. The calibration curves for the three models were overall poorly fitted; DCA showed some net clinical benefit. Model differentiation and net clinical benefit improved by adding three new variables. Based on multivariate analysis, female, age, and maximum tumor diameter ≤ 10 mm, located at the upper pole, Capsular distance < 0mm, US-SLLNM, CLND number ≤ 5 were identified as independent predictors of SLLNM and were used to construct the new model. After 1000 Bootstrap internal validations, the mean AUC of the model was 0.870 (95% CI:0.839-0.901), the calibration curve was close to the ideal curve, and the net clinical benefit was significant. Conclusion: Overall, these nomograms were well differentiated and provided some net clinical benefit, but with varying degrees of underestimation or overestimation of the actual risk and high false-negative rates. New dynamic nomogram was constructed based on the addition of new variables and larger samples, showing better performance.

Optimized personalized management approach for moderate/severe OHSS: development and prospective validation of an OHSS risk assessment index.

STUDY QUESTION: Can a simplified ovarian hyperstimulation syndrome (OHSS) risk assessment index be developed and validated with sufficient discrimination of moderate/severe OHSS from those without OHSS? SUMMARY ANSWER: This easy-to-use OHSS risk assessment index shows good discriminative power and high calibration accuracy in internal and external validation cohorts. WHAT IS KNOWN ALREADY: An early alert and risk stratification is critical to prevent the occurrence of OHSS. We have previously developed a multi-stage smartphone app-based prediction model to evaluate the risk of OHSS, but app use might not be so convenient in many primary institutions. A simplified OHSS risk assessment index has been required. STUDY DESIGN, SIZE, DURATION: This training and internal validation of an OHSS risk assessment index used retrospective cohort data from January 2016 to December 2020. External validation was performed with a prospective cohort database from January 2021 to May 2022. There were 15 066 cycles in the training cohort, 6502 cycles in the internal validation cohort, and 8097 cycles in the external validation cohort. PARTICIPANTS/MATERIALS, SETTING, METHODS: This study was performed in the reproductive medicine center of a tertiary hospital. Infertile women who underwent ovarian stimulation were included. Data were extracted from the local database with detailed medical records. A multi-stage risk assessment index was constructed at multiple stages. The first stage was before the initiation of ovarian stimulation, the second was before the ovulation trigger, the third was after oocyte retrieval, and the last stage was on the embryo transfer day if fresh embryo transfer was scheduled. MAIN RESULTS AND THE ROLE OF CHANCE: We established a simplified multi-stage risk assessment index for moderate/severe OHSS, the performance of which was further evaluated with discrimination and calibration abilities in training and internal and external validation cohorts. The discrimination abilities of the OHSS risk assessment index were determined with C-statistics. C-statistics in training (Stages 1-4: 0.631, 0.692, 0.751, 0.788, respectively) and internal (Stages 1-4: 0.626, 0.642, 0.755, 0.771, respectively) and external validation (Stages 1-4: 0.668, 0.670, 0.754, 0.773, respectively) cohorts were all increased from Stage 1 to 3 with similar trends, and were comparable between Stages 3 and 4. Calibration plots showed high agreement between observed and predicted cases in all three cohorts. Incidences of OHSS based on diverse risk stratification (negligible risk, low risk, medium risk, and high risk) were 0%, 0.6%, 2.7%, and 8.3% in the training cohort, 0%, 0.6%, 3.3%, and 8.5% in the internal validation cohort, and 0.1%, 1.1%, 4.1%, and 7.2% in the external validation cohort. LIMITATIONS, REASONS FOR CAUTION: The influence from clinical interventions including cryopreservation of all embryos cannot be eliminated and thus certain risk factors like estrogen level on trigger day might be assigned with a lower risk score. Another weakness of the study is that several preventive treatments, for instance oral aspirin and letrozole, were not recorded and evaluated in the model. Despite the robust reliability of OHSS assessment index, this tool cannot be used directly for clinical decision-making or as a diagnostic tool. Its value lies in its capacity to evaluate the prognosis of various interventions and to facilitate clinician-patient communication. The combination of this tool and further symptoms and examinations should be all taken into consideration for accurate and personalized management of OHSS. WIDER IMPLICATIONS OF THE FINDINGS: The OHSS risk assessment index can be implemented to facilitate personalized counseling and management of OHSS. STUDY FUNDING/COMPETING INTEREST(S): This study was supported by National Key R&D Program of China (2022YFC2702504), Medical Research Fund Guangdong Provincial (A2024003), and Xinjiang Support Rural Science and Technology (Special Correspondent) Program in Guangdong Province (KTPYJ 2023014). All authors had nothing to disclose. TRIAL REGISTRATION NUMBER: N/A.

[Development and Validation of a Risk Prediction Model for Prolonged Hospitalization in Patients With Diabetic Foot Ulcers]. / ç³–å°¿ç— è¶³æºƒç–¡æ‚£è€ ä½é™¢æ—¶é—´å»¶é•¿é£Žé™©é¢„æµ‹æ¨¡åž‹æž„å»ºåŠéªŒè¯.

Objective: To investigate the risk factors associated with prolonged hospitalization in patients diagnosed with diabetic foot ulcers (DFU), to develop a predictive model, and to conduct internal validation of the model. Methods: The clinical data of DFU patients admitted to West China Hospital, Sichuan University between January 2012 and December 2022 were retrospectively collected. The subjects were randomly assigned to a training cohort and a validation cohort at a ratio of 7 to 3. Hospital stays longer than 75th percentile were defined as prolonged length-of-stay. A thorough analysis of the risk factors was conducted using the training cohort, which enabled the development of an accurate risk prediction model. To ensure robustness, the model was internally validated using the validation cohort. Results: A total of 967 inpatients with DFU were included, among whom 245 patients were identified as having an extended length-of-stay. The training cohort consisted of 622 patients, while the validation cohort comprised 291 patients. Multivariate logistic regression analysis revealed that smoking history (odds ratio [OR]=1.67, 95% confidence interval [CI], 1.13 to 2.48, P=0.010), Wagner grade 3 or higher (OR=7.13, 95% CI, 3.68 to 13.83, P<0.001), midfoot ulcers (OR=1.99, 95% CI, 1.07 to 3.72, P=0.030), posterior foot ulcers (OR=3.68, 95% CI, 1.83 to 7.41, P<0.001), multisite ulcers (OR=2.91, 95% CI, 1.80 to 4.69, P<0.001), wound size≥3 cm2 (OR=2.00, 95% CI, 1.28-3.11, P=0.002), and white blood cell count (OR=1.11, 95% CI, 1.05 to 1.18, P<0.001) were associated with an increased risk of prolonged length of stay. Additionally, a nomogram was constructed based on the identified risk factors. The areas under the receiver operating characteristic (ROC) curves for both the training cohort and the validation cohort were 0.782 (95% CI, 0.745 to 0.820) and 0.756 (95% CI, 0.694 to 0.818), respectively, indicating robust predictive performance. Furthermore, the calibration plot demonstrated optimal concordance between the predicted probabilities and the observed outcomes in both the training and the validation cohorts. Conclusion: Smoking history, Wagner grade≥3, midfoot ulcers, posterior foot ulcers, multisite ulcers, ulcer area≥3 cm2, and elevated white blood cell count are identified as independent predictors of prolonged hospitalization. Therefore, it is imperative that clinicians conduct a comprehensive patient evaluation and implement appropriate diagnostic and therapeutic strategies to effectively shorten the length of stay for DFU patients.

Development and validation of a prediction model for heart failure in patients with heart valvular regurgitation.

AIMS: Patients with heart valvular regurgitation is increasing; early screening of potential patients developing heart failure (HF) is crucial. METHODS: From 1 November 2019 to 31 October 2023, a total of 509 patients with heart valvular regurgitation hospitalized in the Department of Cardiovascular Disease of the First Affiliated Hospital of Guangzhou University of Traditional Medicine were enrolled. Three hundred fifty-six cases were selected as the training set for modelling, and 153 cases were selected as the validation set for the internal validation of the model. RESULTS: A predictive model of heart failure with the following nine risk factors was developed: atrial fibrillation (AF), pulmonary infection (PI), coronary artery disease (CAD), creatinine (CREA), low-density lipoprotein cholesterol (LDL-C), d-dimer (DDi), left ventricular end-diastolic diameter (LVEDd), mitral regurgitation (MR) and aortic regurgitation (AR). The model was evaluated by the C-index [the training set: area under curve (AUC) 0.937, 95% confidence interval (CI) 0.911-0.963; the validation set: AUC 0.928, 95% CI 0.890-0.967]. Hosmer-Lemeshow test (the training set: χ2 10.908, P = 0.207; the validation set: χ2 4.896, P = 0.769) revealed that both the training and validation sets performed well in terms of model differentiation and calibration. Decision curve analysis showed that both the training and validation sets have higher net benefits, indicating that the model has good utility. Ten-fold cross-validation showed that the training set has high similarities with the validation set, which means that the model has good stability. CONCLUSIONS: The occurrence of heart failure in patients with valvular regurgitation has a significant correlation with AF, PI, CAD, CREA, LDL-C, DDi, LVEDd, MR and AR. Based on these risk factors, a prediction model for heart failure was developed and validated, which showed good differentiation and utility, high accuracy and stability, providing a method for predicting heart failure.

Don't be misled: 3 misconceptions about external validation of clinical prediction models.

Clinical prediction models provide risks of health outcomes that can inform patients and support medical decisions. However, most models never make it to actual implementation in practice. A commonly heard reason for this lack of implementation is that prediction models are often not externally validated. While we generally encourage external validation, we argue that an external validation is often neither sufficient nor required as an essential step before implementation. As such, any available external validation should not be perceived as a license for model implementation. We clarify this argument by discussing 3 common misconceptions about external validation. We argue that there is not one type of recommended validation design, not always a necessity for external validation, and sometimes a need for multiple external validations. The insights from this paper can help readers to consider, design, interpret, and appreciate external validation studies.

Development of a rapid LC-MS/MS method for simultaneous quantification of ten commonly used antibiotic drugs in human serum.

The use of TDM in clinical practice to monitor the plasma levels of antibiotics administered to critically ill patients is a well-established approach that allows for optimization of the patient's response to drug therapy, considering the characteristics of the drug, the clinical and physiological status of the patient and any peculiar of the pathogen that caused the clinical picture. In our laboratory, we have developed a single LC-MS/MS analysis for dosing the serum concentration of an antibacterial panel composed of eight antibacterial and two selective inhibitors. The method presented used a certified material furnished by a commercial company and was internally validated using the EMA guidelines. The results have shown high sensitivity, precision, and accuracy, a lower matrix effect combined with simple sample preparation and a time-saving procedure. We have evaluated the recovery rate and matrix effect by testing serum samples without pathological index and serum pools obtained from haemolysed, icteric, or lipemic samples. The assay has shown a recovery range between 94% and 101%.

The development and internal validation of a multivariable model predicting 6-month mortality for people with opioid use disorder presenting to community drug services in England: a protocol.

BACKGROUND: People with opioid use disorder have substantially higher standardised mortality rates compared to the general population; however, lack of clear individual prognostic information presents challenges to prioritise or target interventions within drug treatment services. Previous prognostic models have been developed to estimate the risk of developing opioid use disorder and opioid-related overdose in people routinely prescribed opioids but, to our knowledge, none have been developed to estimate mortality risk in people accessing drug services with opioid use disorder. Initial presentation to drug services is a pragmatic time to evaluate mortality risk given the contemporaneous routine collection of prognostic indicators and as a decision point for appropriate service prioritisation and targeted intervention delivery. This study aims to develop and internally validate a model to estimate 6-month mortality risk for people with opioid use disorder from prognostic indicators recorded at initial assessment in drug services in England. METHODS: An English national dataset containing records from individuals presenting to drug services between 1 April 2013 and 1 April 2023 (n > 800,000) (the National Drug Treatment Monitoring System (NDTMS)) linked to their lifetime hospitalisation and death records (Hospital Episode Statistics-Office of National Statistics (HES-ONS)). Twelve candidate prognostic indicator variables were identified based on literature review of demographic and clinical features associated with increased mortality for people in treatment for opioid use disorder. Variables will be extracted at initial presentation to drug services with mortality measured at 6 months. Two multivariable Cox regression models will be developed one for 6-month all-cause mortality and one for 6-month drug-related mortality using backward elimination with a fractional polynomial approach for continuous variables. Internal validation will be undertaken using bootstrapping methods. Discrimination of both models will be reported using Harrel's c and d-statistics. Calibration curves and slopes will be presented comparing expected and observed event rates. DISCUSSION: The models developed and internally validated in this study aim to improve clinical assessment of mortality risk for people with opioid use disorder presenting to drug services in England. External validation in different populations will be required to develop the model into a tool to assist future clinical decision-making.

COVID-19: The Development and Validation of a New Mortality Risk Score.

Background: The coronavirus disease 2019 (COVID-19) pandemic has found the whole world unprepared for its correct management. Italy was the first European country to experience the spread of the SARS-CoV-2 virus at the end of February 2020. As a result of hospital overcrowding, the quality of care delivered was not always optimal. A substantial number of patients admitted to non-ICU units could have been treated at home. It would have been extremely useful to have a score that, based on personal and clinical characteristics and simple blood tests, could have predicted with sufficient reliability the probability that a patient had or did not have a disease that could have led to their death. This study aims to develop a scoring system to identify which patients with COVID-19 are at high mortality risk upon hospital admission, to expedite and enhance clinical decision making. Methods: A retrospective analysis was performed to develop a multivariable prognostic prediction model. Results: Derivation and external validation cohorts were obtained from two Italian University Hospital databases, including 388 (10.31% deceased) and 1357 (7.68% deceased) patients with confirmed COVID-19, respectively. A multivariable logistic model was used to select seven variables associated with in-hospital death (age, baseline oxygen saturation, hemoglobin value, white blood cell count, percentage of neutrophils, platelet count, and creatinine value). Calibration and discrimination were satisfactory with a cumulative AUC for prediction mortality of 0.924 (95% CI: 0.893-0.944) in derivation cohorts and 0.808 (95% CI: 0.886-0.828) in external validation cohorts. The risk score obtained was compared with the ISARIC 4C Mortality Score, and with all the other most important scores considered so far, to evaluate the risk of death of patients with COVID-19. It performed better than all the above scores to evaluate the predictability of dying. Its sensitivity, specificity, and AUC were higher than the other COVID-19 scoring systems when the latter were calculated for the 388 patients in our derivation cohort. Conclusions: In conclusion, the CZ-COVID-19 Score may help all physicians by identifying those COVID-19 patients who require more attention to provide better therapeutic regimens or, on the contrary, by identifying those patients for whom hospitalization is not necessary and who could therefore be sent home without overcrowding healthcare facilities. We developed and validated a new risk score based on seven variables for upon-hospital admission of COVID-19 patients. It is very simple to calculate and performs better than all the other similar scores to evaluate the predictability of dying.

Surveillance of catheter-associated bloodstream infections: development and validation of a fully automated algorithm.

BACKGROUND: Most surveillance systems for catheter-related bloodstream infections (CRBSI) and central line-associated bloodstream infections (CLABSI) are based on manual chart review. Our objective was to validate a fully automated algorithm for CRBSI and CLABSI surveillance in intensive care units (ICU). METHODS: We developed a fully automated algorithm to detect CRBSI, CLABSI and ICU-onset bloodstream infections (ICU-BSI) in patients admitted to the ICU of a tertiary care hospital in Switzerland. The parameters included in the algorithm were based on a recently performed systematic review. Structured data on demographics, administrative data, central vascular catheter and microbiological results (blood cultures and other clinical cultures) obtained from the hospital's data warehouse were processed by the algorithm. Validation for CRBSI was performed by comparing results with prospective manual BSI surveillance data over a 6-year period. CLABSI were retrospectively assessed over a 2-year period. RESULTS: From January 2016 to December 2021, 854 positive blood cultures were identified in 346 ICU patients. The median age was 61.7 years [IQR 50-70]; 205 (24%) positive samples were collected from female patients. The algorithm detected 5 CRBSI, 109 CLABSI and 280 ICU-BSI. The overall CRBSI and CLABSI incidence rates determined by automated surveillance for the period 2016 to 2021 were 0.18/1000 catheter-days (95% CI 0.06-0.41) and 3.86/1000 catheter days (95% CI: 3.17-4.65). The sensitivity, specificity, positive predictive and negative predictive values of the algorithm for CRBSI, were 83% (95% CI 43.7-96.9), 100% (95% CI 99.5-100), 100% (95% CI 56.5-100), and 99.9% (95% CI 99.2-100), respectively. One CRBSI was misclassified as an ICU-BSI by the algorithm because the same bacterium was identified in the blood culture and in a lower respiratory tract specimen. Manual review of CLABSI from January 2020 to December 2021 (n = 51) did not identify any errors in the algorithm. CONCLUSIONS: A fully automated algorithm for CRBSI and CLABSI detection in critically-ill patients using only structured data provided valid results. The next step will be to assess the feasibility and external validity of implementing it in several hospitals with different electronic health record systems.

A Prediction Model of Preeclampsia in Hyperglycemia Pregnancy.

Purpose: To investigate the risk factors associated with preeclampsia in hyperglycemic pregnancies and develop a predictive model based on routine pregnancy care. Patients and Methods: The retrospective collection of clinical data was performed on 951 pregnant women with hyperglycemia, including those diagnosed with diabetes in pregnancy (DIP) and gestational diabetes mellitus (GDM), who delivered after 34 weeks of gestation at the Maternal and Child Health Hospital Affiliated to Anhui Medical University between January 2017 and December 2019. Observation indicators included liver and kidney function factors testing at 24-29+6 weeks gestation, maternal age, and basal blood pressure. The indicators were screened univariately, and the "rms" package in R language was applied to explore the factors associated with PE in HIP pregnancy by stepwise regression. Multivariable logistic regression analysis was used to develop the prediction model. Based on the above results, a nomogram was constructed to predict the risk of PE occurrence in pregnant women with HIP. Then, the model was evaluated from three aspects: discrimination, calibration, and clinical utility. The internal validation was performed using the bootstrap procedure. Results: Multivariate logistic regression analysis showed that cystatin C, uric acid, glutamyl aminotransferase, blood urea nitrogen, and basal systolic blood pressure as predictors of PE in pregnancy with HIP. The predictive model yielded an area under curve (AUC) value of 0.8031 (95% CI: 0.7383-0.8679), with an optimal threshold of 0.0805, at which point the sensitivity was 0.8307 and specificity of 0.6604. Hosmer-Lemeshow test values were P = 0.3736, Brier score value was 0.0461. After 1000 Bootstrap re-samplings for internal validation, the AUC was 0.7886, the Brier score was 0.0478 and the predicted probability of the calibration curve was similar to the actual probability. A nomogram was constructed based on the above to visualize the model. Conclusion: This study developed a model for predicting PE in pregnant women with HIP, achieving high predictive performance of PE risk through the information of routine pregnancy care.

Internal validation of the GA118-24B Genetic Analyzer, a stable capillary electrophoresis system for forensic DNA identification.

The GA118-24B Genetic Analyzer (hereafter, "GA118-24B") is an independently developed capillary electrophoresis instrument. In the present research, we designed a series of validation experiments to test its performance at detecting DNA fragments compared to the Applied Biosystems 3500 Genetic Analyzer (hereafter, "3500"). Three commercially available autosomal short tandem repeat multiplex kits were used in this validation. The results showed that GA118-24B had acceptable spectral calibration for three kits. The results of accuracy and concordance studies were also satisfactory. GA118-24B showed excellent precision, with a standard deviation of less than 0.1 bp. Sensitivity and mixture studies indicated that GA118-24B could detect low-template DNA and complex mixtures as well as the results generated by 3500 in parallel experiments. Based on the experimental results, we set specific analytical and stochastic thresholds. Besides, GA118-24B showed superiority than 3500 within certain size ranges in the resolution study. Instead of conventional commercial multiplex kits, GA118-24B performed stably on a self-developed eight-dye multiplex system, which were not performed on 3500 Genetic Analyzer. We compared our validation results with those of previous research and found our results to be convincing. Overall, we conclude that GA118-24B is a stable and reliable genetic analyzer for forensic DNA identification.

Performance of the streamlined quality outcomes database web-based calculator: internal and external validation.

BACKGROUND CONTEXT: With an increasing number of web-based calculators designed to provide the probabilities of an individual achieving improvement after lumbar spine surgery, there is a need to determine the accuracy of these models. PURPOSE: To perform an internal and external validation study of the reduced Quality Outcomes Database web-based Calculator (QOD-Calc). STUDY DESIGN: Observational longitudinal cohort. PATIENT SAMPLE: Patients enrolled study-wide in Quality Outcomes Database (QOD) and patients enrolled in DaneSpine at a single institution who had elective lumbar spine surgery with baseline data to complete QOD-Calc and 12-month postoperative data. OUTCOME MEASURES: Oswestry Disability Index (ODI), Numeric Rating Scales (NRS) for back and leg pain, EuroQOL-5D (EQ-5D). METHODS: Baseline data elements were entered into QOD-Calc to determine the probability for each patient having Any Improvement and 30% Improvement in NRS leg pain, back pain, EQ-5D and ODI. These probabilities were compared with the actual 12-month postop data for each of the QOD and DaneSpine cases. Receiver-operating characteristics analyses were performed and calibration plots created to assess model performance. RESULTS: 24,755 QOD cases and 8,105 DaneSpine lumbar cases were included in the analysis. QOD-Calc had acceptable to outstanding ability (AUC: 0.694-0.874) to predict Any Improvement in the QOD cohort and moderate to acceptable ability (AUC: 0.658-0.747) to predict 30% Improvement. QOD-Calc had acceptable to exceptional ability (AUC: 0.669-0.734) to predict Any improvement and moderate to exceptional ability (AUC: 0.619-0.862) to predict 30% Improvement in the DaneSpine cohort. AUCs for the DaneSpine cohort was consistently lower that the AUCs for the QOD validation cohort. CONCLUSION: QOD-Calc performs well in predicting outcomes in a patient population that is similar to the patients that was used to develop it. Although still acceptable, model performance was slightly worse in a distinct population, despite the fact that the sample was more homogenous. Model performance may also be attributed to the low discrimination threshold, with close to 90% of cases reporting Any Improvement in outcome. Prediction models may need to be developed that are highly specific to the characteristics of the population.

Internal validation of reduced PCR reaction volume of the Qiagen Investigator® Argus X-12 QS Kit from blood samples on FTA cards.

The onus of proof in criminal cases is beyond any reasonable doubt, and the issue on the lack of complete internal validation data can be manipulated when it comes to justifying the validity and reliability of the X-chromosomal short tandem repeats analysis for court representation. Therefore, this research evaluated the efficiency of the optimized 60% reduced volumes for polymerase chain reaction (PCR) amplification using the Qiagen Investigator® Argus X-12 QS Kit, as well as the capillary electrophoresis (CE) sample preparation for blood samples on Flinder's Technology Associates (FTA) cards. Good-quality DNA profile (3000-12,000 RFU) from the purified blood sample on FTA card (1.2 mm) were obtained using the optimized PCR (10.0 µL of PCR reaction volume and 21 cycles) and CE (9.0 µL Hi-Di™ Formamide and 0.3 µL DNA Size Standard 550 [BTO] and 27 s injection time) conditions. The analytical and stochastic thresholds were 100 and 200 RFU, respectively. Hence, the internal validation data supported the use of the optimized 60% reduced PCR amplification reaction volume of the Qiagen Investigator® Argus X-12 QS Kit as well as the CE sample preparation for producing reliable DNA profiles that comply with the quality assurance standards for forensic DNA testing laboratories, while optimizing the analytical cost.

A Validated Composite Score Demonstrates Potential Superiority to MELD-Based Systems in Predicting Short-Term Survival in Patients with Liver Cirrhosis and Spontaneous Bacterial Peritonitis-A Preliminary Study.

BACKGROUND: Spontaneous bacterial peritonitis (SBP) is a severe complication in cirrhosis patients with ascites, leading to high mortality rates if not promptly treated. However, specific prediction models for SBP are lacking. AIMS: This study aimed to compare commonly used cirrhotic prediction models (CTP score, MELD, MELD-Na, iMELD, and MELD 3.0) for short-term mortality prediction and develop a novel model to improve mortality prediction. METHODS: Patients with the first episode of SBP were included. Prognostic values for mortality were assessed using AUROC analysis. A novel prediction model was developed and validated. RESULTS: In total, 327 SBP patients were analyzed, with HBV infection as the main etiologies. MELD 3.0 demonstrated the highest AUROC among the traditional models. The novel model, incorporating HRS, exhibited superior predictive accuracy for in-hospital in all patients and 3-month mortality in HBV-cirrhosis, with AUROC values of 0.827 and 0.813 respectively, surpassing 0.8. CONCLUSIONS: MELD 3.0 score outperformed the CTP score and showed a non-significant improvement compared to other MELD-based scores, while the novel SBP model demonstrated impressive accuracy. Internal validation and an HBV-related cirrhosis subgroup sensitivity analysis supported these findings, highlighting the need for a specific prognostic model for SBP and the importance of preventing HRS development to improve SBP prognosis.

Quality indices for topic model selection and evaluation: a literature review and case study.

BACKGROUND: Topic models are a class of unsupervised machine learning models, which facilitate summarization, browsing and retrieval from large unstructured document collections. This study reviews several methods for assessing the quality of unsupervised topic models estimated using non-negative matrix factorization. Techniques for topic model validation have been developed across disparate fields. We synthesize this literature, discuss the advantages and disadvantages of different techniques for topic model validation, and illustrate their usefulness for guiding model selection on a large clinical text corpus. DESIGN, SETTING AND DATA: Using a retrospective cohort design, we curated a text corpus containing 382,666 clinical notes collected between 01/01/2017 through 12/31/2020 from primary care electronic medical records in Toronto Canada. METHODS: Several topic model quality metrics have been proposed to assess different aspects of model fit. We explored the following metrics: reconstruction error, topic coherence, rank biased overlap, Kendall's weighted tau, partition coefficient, partition entropy and the Xie-Beni statistic. Depending on context, cross-validation and/or bootstrap stability analysis were used to estimate these metrics on our corpus. RESULTS: Cross-validated reconstruction error favored large topic models (K ≥ 100 topics) on our corpus. Stability analysis using topic coherence and the Xie-Beni statistic also favored large models (K = 100 topics). Rank biased overlap and Kendall's weighted tau favored small models (K = 5 topics). Few model evaluation metrics suggested mid-sized topic models (25 ≤ K ≤ 75) as being optimal. However, human judgement suggested that mid-sized topic models produced expressive low-dimensional summarizations of the corpus. CONCLUSIONS: Topic model quality indices are transparent quantitative tools for guiding model selection and evaluation. Our empirical illustration demonstrated that different topic model quality indices favor models of different complexity; and may not select models aligning with human judgment. This suggests that different metrics capture different aspects of model goodness of fit. A combination of topic model quality indices, coupled with human validation, may be useful in appraising unsupervised topic models.

Development and internal validation of a non-invasive clinical tool to predict sufficient omega-3 levels in early pregnancy.

BACKGROUND: Complications from preterm birth (PTB) are the leading cause of death and disability in those under five years. Whilst the role of omega-3 (n-3) supplementation in reducing PTB is well-established, growing evidence suggests supplementation use in those replete may increase the risk of early PTB. AIM: To develop a non-invasive tool to identify individuals with total n-3 serum levels above 4.3% of total fatty acids in early pregnancy. METHODS: We conducted a prospective observational study recruiting 331 participants from three clinical sites in Newcastle, Australia. Eligible participants (n = 307) had a singleton pregnancy between 8 and 20 weeks' gestation at recruitment. Data on factors associated with n-3 serum levels were collected using an electronic questionnaire; these included estimated intake of n-3 (including food type, portion size, frequency of consumption), n-3 supplementation, and sociodemographic factors. The optimal cut-point of estimated n-3 intake that predicted mothers with total serum n-3 levels likely above 4.3% was developed using multivariate logistic regression, adjusting for maternal age, body mass index, socioeconomic status, and n-3 supplementation use. Total serum n-3 levels above 4.3% was selected as previous research has demonstrated that mothers with these levels are at increased risk of early PTB if they take additional n-3 supplementation during pregnancy. Models were evaluated using various performance metrics including sensitivity, specificity, area under receiver operator characteristic (AUROC) curve, true positive rate (TPR) at 10% false positive rate (FPR), Youden Index, Closest to (0,1) Criteria, Concordance Probability, and Index of Union. Internal validation was performed using 1000-bootstraps to generate 95% confidence intervals for performance metrics generated. RESULTS: Of 307 eligible participants included for analysis, 58.6% had total n-3 serum levels above 4.3%. The optimal model had a moderate discriminative ability (AUROC 0.744, 95% CI 0.742-0.746) with 84.7% sensitivity, 54.7% specificity and 37.6% TPR at 10% FPR. CONCLUSIONS: Our non-invasive tool was a moderate predictor of pregnant women with total serum n-3 levels above 4.3%; however, its performance is not yet adequate for clinical use. TRIAL REGISTRATION: This trial was approved by the Hunter New England Human Research Ethics Committee of the Hunter New England Local Health District (Reference 2020/ETH00498 on 07/05/2020 and 2020/ETH02881 on 08/12/2020).

Development and internal validation of a model to predict long-term survival of ANCA associated vasculitis.

Objectives: Risk stratification and prognosis prediction are critical for appropriate management of anti-neutrophil cytoplasmic antibody (ANCA) associated vasculitis (AAV). Herein, we aim to develop and internally validate a prediction model specifically for long-term survival of patients with AAV. Methods: We thoroughly reviewed the medical charts of patients with AAV admitted to Peking Union Medical College Hospital from January 1999 to July 2019. The Least Absolute Shrinkage and Selection Operator method and the COX proportional hazard regression was used to develop the prediction model. The Harrell's concordance index (C-index), calibration curves and Brier scores were calculated to evaluate the model performance. The model was internally validated by bootstrap resampling methods. Results: A total of 653 patients were included in the study, including 303 patients with microscopic polyangiitis, 245 patients with granulomatosis with polyangiitis and 105 patients with eosinophilic granulomatosis with polyangiitis, respectively. During a median follow-up of 33 months (interquartile range 15-60 months), 120 deaths occurred. Age at admission, chest and cardiovascular involvement, serum creatinine grade, hemoglobin levels at baseline and AAV sub-types were selected as predictive parameters in the final model. The optimism-corrected C-index and integrated Brier score of our prediction model were 0.728 and 0.109. The calibration plots showed fine agreement between observed and predicted probability of all-cause death. The decision curve analysis (DCA) showed that in a wide range of threshold probabilities, our prediction model had higher net benefits compared with the revised five factor score (rFFSand) and the birmingham vasculitis activity score (BVAS) system. Conclusion: Our model performs well in predicting outcomes of AAV patients. Patients with moderate-to-high probability of death should be followed closely and personalized monitoring plan should be scheduled.

Application of artificial intelligence for overall survival risk stratification in oropharyngeal carcinoma: A validation of ProgTOOL.

BACKGROUND: In recent years, there has been a surge in machine learning-based models for diagnosis and prognostication of outcomes in oncology. However, there are concerns relating to the model's reproducibility and generalizability to a separate patient cohort (i.e., external validation). OBJECTIVES: This study primarily provides a validation study for a recently introduced and publicly available machine learning (ML) web-based prognostic tool (ProgTOOL) for overall survival risk stratification of oropharyngeal squamous cell carcinoma (OPSCC). Additionally, we reviewed the published studies that have utilized ML for outcome prognostication in OPSCC to examine how many of these models were externally validated, type of external validation, characteristics of the external dataset, and diagnostic performance characteristics on the internal validation (IV) and external validation (EV) datasets were extracted and compared. METHODS: We used a total of 163 OPSCC patients obtained from the Helsinki University Hospital to externally validate the ProgTOOL for generalizability. In addition, PubMed, OvidMedline, Scopus, and Web of Science databases were systematically searched according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. RESULTS: The ProgTOOL produced a predictive performance of 86.5% balanced accuracy, Mathew's correlation coefficient of 0.78, Net Benefit (0.7) and Brier score (0.06) for overall survival stratification of OPSCC patients as either low-chance or high-chance. In addition, out of a total of 31 studies found to have used ML for the prognostication of outcomes in OPSCC, only seven (22.6%) reported a form of EV. Three studies (42.9%) each used either temporal EV or geographical EV while only one study (14.2%) used expert as a form of EV. Most of the studies reported a reduction in performance when externally validated. CONCLUSION: The performance of the model in this validation study indicates that it may be generalized, therefore, bringing recommendations of the model for clinical evaluation closer to reality. However, the number of externally validated ML-based models for OPSCC is still relatively small. This significantly limits the transfer of these models for clinical evaluation and subsequently reduces the likelihood of the use of these models in daily clinical practice. As a gold standard, we recommend the use of geographical EV and validation studies to reveal biases and overfitting of these models. These recommendations are poised to facilitate the implementation of these models in clinical practice.

Predictive models to assess risk of extended length of stay in adults with spinal deformity and lumbar degenerative pathology: development and internal validation.

BACKGROUND CONTEXT: Postoperative recovery after adult spinal deformity (ASD) operations is arduous, fraught with complications, and often requires extended hospital stays. A need exists for a method to rapidly predict patients at risk for extended length of stay (eLOS) in the preoperative setting. PURPOSE: To develop a machine learning model to preoperatively estimate the likelihood of eLOS following elective multi-level lumbar/thoracolumbar spinal instrumented fusions (≥3 segments) for ASD. STUDY DESIGN/SETTING: Retrospectively from a state-level inpatient database hosted by the Health care cost and Utilization Project. PATIENT SAMPLE: Of 8,866 patients of age ≥50 with ASD undergoing elective lumbar or thoracolumbar multilevel instrumented fusions. OUTCOME MEASURES: The primary outcome was eLOS (>7 days). METHODS: Predictive variables consisted of demographics, comorbidities, and operative information. Significant variables from univariate and multivariate analyses were used to develop a logistic regression-based predictive model that use six predictors. Model accuracy was assessed through area under the curve (AUC), sensitivity, and specificity. RESULTS: Of 8,866 patients met inclusion criteria. A saturated logistic model with all significant variables from multivariate analysis was developed (AUC=0.77), followed by generation of a simplified logistic model through stepwise logistic regression (AUC=0.76). Peak AUC was reached with inclusion of six selected predictors (combined anterior and posterior approach, surgery to both lumbar and thoracic regions, ≥8 level fusion, malnutrition, congestive heart failure, and academic institution). A cutoff of 0.18 for eLOS yielded a sensitivity of 77% and specificity of 68%. CONCLUSIONS: This predictive model can facilitate identification of adults at risk for eLOS following elective multilevel lumbar/thoracolumbar spinal instrumented fusions for ASD. With a fair diagnostic accuracy, the predictive calculator will ideally enable clinicians to improve preoperative planning, guide patient expectations, enable optimization of modifiable risk factors, facilitate appropriate discharge planning, stratify financial risk, and accurately identify patients who may represent high-cost outliers. Future prospective studies that validate this risk assessment tool on external datasets would be valuable.

There is no such thing as a validated prediction model.

BACKGROUND: Clinical prediction models should be validated before implementation in clinical practice. But is favorable performance at internal validation or one external validation sufficient to claim that a prediction model works well in the intended clinical context? MAIN BODY: We argue to the contrary because (1) patient populations vary, (2) measurement procedures vary, and (3) populations and measurements change over time. Hence, we have to expect heterogeneity in model performance between locations and settings, and across time. It follows that prediction models are never truly validated. This does not imply that validation is not important. Rather, the current focus on developing new models should shift to a focus on more extensive, well-conducted, and well-reported validation studies of promising models. CONCLUSION: Principled validation strategies are needed to understand and quantify heterogeneity, monitor performance over time, and update prediction models when appropriate. Such strategies will help to ensure that prediction models stay up-to-date and safe to support clinical decision-making.