Retrospective validation study of an artificial neural network-based preoperative decision-support tool for noninvasive lymph node staging (NILS) in women with primary breast cancer (ISRCTN14341750).

BACKGROUND: Surgical sentinel lymph node biopsy (SLNB) is routinely used to reliably stage axillary lymph nodes in early breast cancer (BC). However, SLNB may be associated with postoperative arm morbidities. For most patients with BC undergoing SLNB, the findings are benign, and the procedure is currently questioned. A decision-support tool for the prediction of benign sentinel lymph nodes based on preoperatively available data has been developed using artificial neural network modelling. METHODS: This was a retrospective geographical and temporal validation study of the noninvasive lymph node staging (NILS) model, based on preoperatively available data from 586 women consecutively diagnosed with primary BC at two sites. Ten preoperative clinicopathological characteristics from each patient were entered into the web-based calculator, and the probability of benign lymph nodes was predicted. The performance of the NILS model was assessed in terms of discrimination with the area under the receiver operating characteristic curve (AUC) and calibration, that is, comparison of the observed and predicted event rates of benign axillary nodal status (N0) using calibration slope and intercept. The primary endpoint was axillary nodal status (discrimination, benign [N0] vs. metastatic axillary nodal status [N+]) determined by the NILS model compared to nodal status by definitive pathology. RESULTS: The mean age of the women in the cohort was 65 years, and most of them (93%) had luminal cancers. Approximately three-fourths of the patients had no metastases in SLNB (N0 74% and 73%, respectively). The AUC for the predicted probabilities for the whole cohort was 0.6741 (95% confidence interval: 0.6255-0.7227). More than one in four patients (n = 151, 26%) were identified as candidates for SLNB omission when applying the predefined cut-off for lymph node-negative status from the development cohort. The NILS model showed the best calibration in patients with a predicted high probability of healthy axilla. CONCLUSION: The performance of the NILS model was satisfactory. In approximately every fourth patient, SLNB could potentially be omitted. Considering the shift from postoperatively to preoperatively available predictors in this validation study, we have demonstrated the robustness of the NILS model. The clinical usability of the web interface will be evaluated before its clinical implementation. TRIAL REGISTRATION: Registered in the ISRCTN registry with study ID ISRCTN14341750. Date of registration 23/11/2018.

Prior electrocardiograms not useful for machine learning predictions of major adverse cardiac events in emergency department chest pain patients.

At the emergency department (ED), it is important to quickly and accurately determine which patients are likely to have a major adverse cardiac event (MACE). Machine learning (ML) models can be used to aid physicians in detecting MACE, and improving the performance of such models is an active area of research. In this study, we sought to determine if ML models can be improved by including a prior electrocardiogram (ECG) from each patient. To that end, we trained several models to predict MACE within 30 days, both with and without prior ECGs, using data collected from 19,499 consecutive patients with chest pain, from five EDs in southern Sweden, between the years 2017 and 2018. Our results indicate no improvement in AUC from prior ECGs. This was consistent across models, both with and without additional clinical input variables, for different patient subgroups, and for different subsets of the outcome. While contradicting current best practices for manual ECG analysis, the results are positive in the sense that ML models with fewer inputs are more easily and widely applicable in practice.

Deep learning prediction models based on EHR trajectories: A systematic review.

BACKGROUND: Electronic health records (EHRs) are generated at an ever-increasing rate. EHR trajectories, the temporal aspect of health records, facilitate predicting patients' future health-related risks. It enables healthcare systems to increase the quality of care through early identification and primary prevention. Deep learning techniques have shown great capacity for analyzing complex data and have been successful for prediction tasks using complex EHR trajectories. This systematic review aims to analyze recent studies to identify challenges, knowledge gaps, and ongoing research directions. METHODS: For this systematic review, we searched Scopus, PubMed, IEEE Xplore, and ACM databases from Jan 2016 to April 2022 using search terms centered around EHR, deep learning, and trajectories. Then the selected papers were analyzed according to publication characteristics, objectives, and their solutions regarding existing challenges, such as the model's capacity to deal with intricate data dependencies, data insufficiency, and explainability. RESULTS: After removing duplicates and out-of-scope papers, 63 papers were selected, which showed rapid growth in the number of research in recent years. Predicting all diseases in the next visit and the onset of cardiovascular diseases were the most common targets. Different contextual and non-contextual representation learning methods are employed to retrieve important information from the sequence of EHR trajectories. Recurrent neural networks and the time-aware attention mechanism for modeling long-term dependencies, self-attentions, convolutional neural networks, graphs for representing inner visit relations, and attention scores for explainability were frequently used among the reviewed publications. CONCLUSIONS: This systematic review demonstrated how recent breakthroughs in deep learning methods have facilitated the modeling of EHR trajectories. Research on improving the ability of graph neural networks, attention mechanisms, and cross-modal learning to analyze intricate dependencies among EHRs has shown good progress. There is a need to increase the number of publicly available EHR trajectory datasets to allow for easier comparison among different models. Also, very few developed models can handle all aspects of EHR trajectory data.

Machine learning for early prediction of acute myocardial infarction or death in acute chest pain patients using electrocardiogram and blood tests at presentation.

AIMS: In the present study, we aimed to evaluate the performance of machine learning (ML) models for identification of acute myocardial infarction (AMI) or death within 30 days among emergency department (ED) chest pain patients. METHODS AND RESULTS: Using data from 9519 consecutive ED chest pain patients, we created ML models based on logistic regression or artificial neural networks. Model inputs included sex, age, ECG and the first blood tests at patient presentation: High sensitivity TnT (hs-cTnT), glucose, creatinine, and hemoglobin. For a safe rule-out, the models were adapted to achieve a sensitivity > 99% and a negative predictive value (NPV) > 99.5% for 30-day AMI/death. For rule-in, we set the models to achieve a specificity > 90% and a positive predictive value (PPV) of > 70%. The models were also compared with the 0 h arm of the European Society of Cardiology algorithm (ESC 0 h); An initial hs-cTnT < 5 ng/L for rule-out and ≥ 52 ng/L for rule-in. A convolutional neural network was the best model and identified 55% of the patients for rule-out and 5.3% for rule-in, while maintaining the required sensitivity, specificity, NPV and PPV levels. ESC 0 h failed to reach these performance levels. DISCUSSION: An ML model based on age, sex, ECG and blood tests at ED arrival can identify six out of ten chest pain patients for safe early rule-out or rule-in with no need for serial blood tests. Future studies should attempt to improve these ML models further, e.g. by including additional input data.

The implementation of a noninvasive lymph node staging (NILS) preoperative prediction model is cost effective in primary breast cancer.

PURPOSE: The need for sentinel lymph node biopsy (SLNB) in clinically node-negative (cN0) patients is currently questioned. Our objective was to investigate the cost-effectiveness of a preoperative noninvasive lymph node staging (NILS) model (an artificial neural network model) for predicting pathological nodal status in patients with cN0 breast cancer (BC). METHODS: A health-economic decision-analytic model was developed to evaluate the utility of the NILS model in reducing the proportion of cN0 patients with low predicted risk undergoing SLNB. The model used information from a national registry and published studies, and three sensitivity/specificity scenarios of the NILS model were evaluated. Subgroup analysis explored the outcomes of breast-conserving surgery (BCS) or mastectomy. The results are presented as cost (€) and quality-adjusted life years (QALYs) per 1000 patients. RESULTS: All three scenarios of the NILS model reduced total costs (-€93,244 to -€398,941 per 1000 patients). The overall health benefit allowing for the impact of SLNB complications was a net health gain (7.0-26.9 QALYs per 1000 patients). Sensitivity analyses disregarding reduced quality of life from lymphedema showed a small loss in total health benefits (0.4-4.0 QALYs per 1000 patients) because of the reduction in total life years (0.6-6.5 life years per 1000 patients) after reduced adjuvant treatment. Subgroup analyses showed greater cost reductions and QALY gains in patients undergoing BCS. CONCLUSION: Implementing the NILS model to identify patients with low risk for nodal metastases was associated with substantial cost reductions and likely overall health gains, especially in patients undergoing BCS.

Proteomic Data Analysis for Differential Profiling of the Autoimmune Diseases SLE, RA, SS, and ANCA-Associated Vasculitis.

Early and correct diagnosis of inflammatory rheumatic diseases (IRD) poses a clinical challenge due to the multifaceted nature of symptoms, which also may change over time. The aim of this study was to perform protein expression profiling of four systemic IRDs, systemic lupus erythematosus (SLE), ANCA-associated systemic vasculitis (SV), rheumatoid arthritis (RA), and Sjögren's syndrome (SS), and healthy controls to identify candidate biomarker signatures for differential classification. A total of 316 serum samples collected from patients with SLE, RA, SS, or SV and from healthy controls were analyzed using 394-plex recombinant antibody microarrays. Differential protein expression profiling was examined using Wilcoxon signed rank test, and condensed biomarker panels were identified using advanced bioinformatics and state-of-the art classification algorithms to pinpoint signatures reflecting each disease (raw data set available at https://figshare.com/s/3bd3848a28ef6e7ae9a9.). In this study, we were able to classify the included individual IRDs with high accuracy, as demonstrated by the ROC area under the curve (ROC AUC) values ranging between 0.96 and 0.80. In addition, the groups of IRDs could be separated from healthy controls at an ROC AUC value of 0.94. Disease-specific candidate biomarker signatures and general autoimmune signature were identified, including several deregulated analytes. This study supports the rationale of using multiplexed affinity-based technologies to reflect the biological complexity of autoimmune diseases. A multiplexed approach for decoding multifactorial complex diseases, such as autoimmune diseases, will play a significant role for future diagnostic purposes, essential to prevent severe organ- and tissue-related damage.

Improving Machine Learning 30-Day Mortality Prediction by Discounting Surprising Deaths.

BACKGROUND: Machine learning (ML) is an emerging tool for predicting need of end-of-life discussion and palliative care, by using mortality as a proxy. But deaths, unforeseen by emergency physicians at time of the emergency department (ED) visit, might have a weaker association with the ED visit. OBJECTIVES: To develop an ML algorithm that predicts unsurprising deaths within 30 days after ED discharge. METHODS: In this retrospective registry study, we included all ED attendances within the Swedish region of Halland in 2015 and 2016. All registered deaths within 30 days after ED discharge were classified as either "surprising" or "unsurprising" by an adjudicating committee with three senior specialists in emergency medicine. ML algorithms were developed for the death subclasses by using Logistic Regression (LR), Random Forest (RF), and Support Vector Machine (SVM). RESULTS: Of all 30-day deaths (n = 148), 76% (n = 113) were not surprising to the adjudicating committee. The most common diseases were advanced stage cancer, multidisease/frailty, and dementia. By using LR, RF, and SVM, mean area under the receiver operating characteristic curve (ROC-AUC) of unsurprising deaths in the test set were 0.950 (SD 0.008), 0.944 (SD 0.007), and 0.949 (SD 0.007), respectively. For all mortality, the ROC-AUCs for LR, RF, and SVM were 0.924 (SD 0.012), 0.922 (SD 0.009), and 0.931 (SD 0.008). The difference in prediction performance between all and unsurprising death was statistically significant (P < .001) for all three models. CONCLUSION: In patients discharged to home from the ED, three-quarters of all 30-day deaths did not surprise an adjudicating committee with emergency medicine specialists. When only unsurprising deaths were included, ML mortality prediction improved significantly.

Predicting and elucidating the etiology of fatty liver disease: A machine learning modeling and validation study in the IMI DIRECT cohorts.

BACKGROUND: Non-alcoholic fatty liver disease (NAFLD) is highly prevalent and causes serious health complications in individuals with and without type 2 diabetes (T2D). Early diagnosis of NAFLD is important, as this can help prevent irreversible damage to the liver and, ultimately, hepatocellular carcinomas. We sought to expand etiological understanding and develop a diagnostic tool for NAFLD using machine learning. METHODS AND FINDINGS: We utilized the baseline data from IMI DIRECT, a multicenter prospective cohort study of 3,029 European-ancestry adults recently diagnosed with T2D (n = 795) or at high risk of developing the disease (n = 2,234). Multi-omics (genetic, transcriptomic, proteomic, and metabolomic) and clinical (liver enzymes and other serological biomarkers, anthropometry, measures of beta-cell function, insulin sensitivity, and lifestyle) data comprised the key input variables. The models were trained on MRI-image-derived liver fat content (<5% or ≥5%) available for 1,514 participants. We applied LASSO (least absolute shrinkage and selection operator) to select features from the different layers of omics data and random forest analysis to develop the models. The prediction models included clinical and omics variables separately or in combination. A model including all omics and clinical variables yielded a cross-validated receiver operating characteristic area under the curve (ROCAUC) of 0.84 (95% CI 0.82, 0.86; p < 0.001), which compared with a ROCAUC of 0.82 (95% CI 0.81, 0.83; p < 0.001) for a model including 9 clinically accessible variables. The IMI DIRECT prediction models outperformed existing noninvasive NAFLD prediction tools. One limitation is that these analyses were performed in adults of European ancestry residing in northern Europe, and it is unknown how well these findings will translate to people of other ancestries and exposed to environmental risk factors that differ from those of the present cohort. Another key limitation of this study is that the prediction was done on a binary outcome of liver fat quantity (<5% or ≥5%) rather than a continuous one. CONCLUSIONS: In this study, we developed several models with different combinations of clinical and omics data and identified biological features that appear to be associated with liver fat accumulation. In general, the clinical variables showed better prediction ability than the complex omics variables. However, the combination of omics and clinical variables yielded the highest accuracy. We have incorporated the developed clinical models into a web interface (see: https://www.predictliverfat.org/) and made it available to the community. TRIAL REGISTRATION: ClinicalTrials.gov NCT03814915.

Artificial neural network models to predict nodal status in clinically node-negative breast cancer.

BACKGROUND: Sentinel lymph node biopsy (SLNB) is standard staging procedure for nodal status in breast cancer, but lacks therapeutic benefit for patients with benign sentinel nodes. For patients with positive sentinel nodes, individualized surgical strategies are applied depending on the extent of nodal involvement. Preoperative prediction of nodal status is thus important for individualizing axillary surgery avoiding unnecessary surgery. We aimed to predict nodal status in clinically node-negative breast cancer and identify candidates for SLNB omission by including patient-related and pathological characteristics into artificial neural network (ANN) models. METHODS: Patients with primary breast cancer were consecutively included between January 1, 2009 and December 31, 2012 in a prospectively maintained pathology database. Clinical- and radiological data were extracted from patient's files and only clinically node-negative patients constituted the final study cohort. ANN-based models for nodal prediction were constructed including 15 risk variables for nodal status. Area under the receiver operating characteristic curve (AUC) and Hosmer-Lemeshow goodness-of-fit test (HL) were used to assess performance and calibration of three predictive ANN-based models for no lymph node metastasis (N0), metastases in 1-3 lymph nodes (N1) and metastases in ≥ 4 lymph nodes (N2). Linear regression models for nodal prediction were calculated for comparison. RESULTS: Eight hundred patients (N0, n = 514; N1, n = 232; N2, n = 54) were included. Internally validated AUCs for N0 versus N+ was 0.740 (95% CI = 0.723-0.758); median HL was 9.869 (P = 0.274), for N1 versus N0, 0.705 (95% CI = 0.686-0.724; median HL: 7.421; P = 0.492) and for N2 versus N0 and N1, 0.747 (95% CI = 0.728-0.765; median HL: 9.220; P = 0.324). Tumor size and vascular invasion were top-ranked predictors of all three end-points, followed by estrogen receptor status and lobular cancer for prediction of N2. For each end-point, ANN models showed better discriminatory performance than multivariable logistic regression models. Accepting a false negative rate (FNR) of 10% for predicting N0 by the ANN model, SLNB could have been abstained in 27.25% of patients with clinically node-negative axilla. CONCLUSIONS: In this retrospective study, ANN showed promising result as decision-supporting tools for estimating nodal disease. If prospectively validated, patients least likely to have nodal metastasis could be spared SLNB using predictive models. TRIAL REGISTRATION: Registered in the ISRCTN registry with study ID ISRCTN14341750 . Date of registration 23/11/2018. Retrospectively registered.

Transcriptional regulation of lineage commitment--a stochastic model of cell fate decisions.

Molecular mechanisms employed by individual multipotent cells at the point of lineage commitment remain largely uncharacterized. Current paradigms span from instructive to noise-driven mechanisms. Of considerable interest is also whether commitment involves a limited set of genes or the entire transcriptional program, and to what extent gene expression configures multiple trajectories into commitment. Importantly, the transient nature of the commitment transition confounds the experimental capture of committing cells. We develop a computational framework that simulates stochastic commitment events, and affords mechanistic exploration of the fate transition. We use a combined modeling approach guided by gene expression classifier methods that infers a time-series of stochastic commitment events from experimental growth characteristics and gene expression profiling of individual hematopoietic cells captured immediately before and after commitment. We define putative regulators of commitment and probabilistic rules of transition through machine learning methods, and employ clustering and correlation analyses to interrogate gene regulatory interactions in multipotent cells. Against this background, we develop a Monte Carlo time-series stochastic model of transcription where the parameters governing promoter status, mRNA production and mRNA decay in multipotent cells are fitted to experimental static gene expression distributions. Monte Carlo time is converted to physical time using cell culture kinetic data. Probability of commitment in time is a function of gene expression as defined by a logistic regression model obtained from experimental single-cell expression data. Our approach should be applicable to similar differentiating systems where single cell data is available. Within our system, we identify robust model solutions for the multipotent population within physiologically reasonable values and explore model predictions with regard to molecular scenarios of entry into commitment. The model suggests distinct dependencies of different commitment-associated genes on mRNA dynamics and promoter activity, which globally influence the probability of lineage commitment.

Analysis of regional bone scan index measurements for the survival of patients with prostate cancer.

BACKGROUND: A bone scan is a common method for monitoring bone metastases in patients with advanced prostate cancer. The Bone Scan Index (BSI) measures the tumor burden on the skeleton, expressed as a percentage of the total skeletal mass. Previous studies have shown that BSI is associated with survival of prostate cancer patients. The objective in this study was to investigate to what extent regional BSI measurements, as obtained by an automated method, can improve the survival analysis for advanced prostate cancer. METHODS: The automated method for analyzing bone scan images computed BSI values for twelve skeletal regions, in a study population consisting of 1013 patients diagnosed with prostate cancer. In the survival analysis we used the standard Cox proportional hazards model and a more advanced non-linear method based on artificial neural networks. The concordance index (C-index) was used to measure the performance of the models. RESULTS: A Cox model with age and total BSI obtained a C-index of 70.4%. The best Cox model with regional measurements from Costae, Pelvis, Scapula and the Spine, together with age, got a similar C-index (70.5%). The overall best single skeletal localisation, as measured by the C-index, was Costae. The non-linear model performed equally well as the Cox model, ruling out any significant non-linear interactions among the regional BSI measurements. CONCLUSION: The present study showed that the localisation of bone metastases obtained from the bone scans in prostate cancer patients does not improve the performance of the survival models compared to models using the total BSI. However a ranking procedure indicated that some regions are more important than others.

Molecular serum portraits in patients with primary breast cancer predict the development of distant metastases.

The risk of distant recurrence in breast cancer patients is difficult to assess with current clinical and histopathological parameters, and no validated serum biomarkers currently exist. Using a recently developed recombinant antibody microarray platform containing 135 antibodies against 65 mainly immunoregulatory proteins, we screened 240 sera from 64 patients with primary breast cancer. This unique longitudinal sample material was collected from each patient between 0 and 36 mo after the primary operation. The velocity for each serum protein was determined by comparing the samples collected at the primary operation and then 3-6 mo later. A 21-protein signature was identified, using leave-one-out cross-validation together with a backward elimination strategy in a training cohort. This signature was tested and evaluated subsequently in an independent test cohort (prevalidation). The risk of developing distant recurrence after primary operation could be assessed for each patient, using her molecular portraits. The results from this prevalidation study showed that patients could be classified into high- versus low-risk groups for developing metastatic breast cancer with a receiver operating characteristic area under the curve of 0.85. This risk assessment was not dependent on the type of adjuvant therapy received by the patients. Even more importantly, we demonstrated that this protein signature provided an added value compared with conventional clinical parameters. Consequently, we present here a candidate serum biomarker signature able to classify patients with primary breast cancer according to their risk of developing distant recurrence, with an accuracy outperforming current procedures.

The Concordance Index decomposition: A measure for a deeper understanding of survival prediction models.

The Concordance Index (C-index) is a commonly used metric in Survival Analysis for evaluating the performance of a prediction model. In this paper, we propose a decomposition of the C-index into a weighted harmonic mean of two quantities: one for ranking observed events versus other observed events, and the other for ranking observed events versus censored cases. This decomposition enables a finer-grained analysis of the relative strengths and weaknesses between different survival prediction methods. The usefulness of this decomposition is demonstrated through benchmark comparisons against classical models and state-of-the-art methods, together with the new variational generative neural-network-based method (SurVED) proposed in this paper. The performance of the models is assessed using four publicly available datasets with varying levels of censoring. Using the C-index decomposition and synthetic censoring, the analysis shows that deep learning models utilize the observed events more effectively than other models. This allows them to keep a stable C-index in different censoring levels. In contrast to such deep learning methods, classical machine learning models deteriorate when the censoring level decreases due to their inability to improve on ranking the events versus other events.

Human bias and CNNs' superior insights in satellite based poverty mapping.

Satellite imagery is a potent tool for estimating human wealth and poverty, especially in regions lacking reliable data. This study compares a range of poverty estimation approaches from satellite images, spanning from expert-based to fully machine learning-based methodologies. Human experts ranked clusters from the Tanzania DHS survey using high-resolution satellite images. Then expert-defined features were utilized in a machine learning algorithm to estimate poverty. An explainability method was applied to assess the importance and interaction of these features in poverty prediction. Additionally, a convolutional neural network (CNN) was employed to estimate poverty from medium-resolution satellite images of the same locations. Our analysis indicates that increased human involvement in poverty estimation diminishes accuracy compared to machine learning involvement, exemplified with the case of Tanzania. Expert defined features exhibited significant overlap and poor interaction when used together in a classifier. Conversely, the CNN-based approach outperformed human experts, demonstrating superior predictive capability with medium-resolution images. These findings highlight the importance of leveraging machine learning explainability methods to identify predictive elements that may be overlooked by human experts. This study advocates for the integration of emerging technologies with traditional methodologies to optimize data collection and analysis of poverty and welfare.

Surveillance of Disease Outbreaks Using Unsupervised Uni-Multivariate Anomaly Detection of Time-Series Symptoms.

Effectively identifying deviations in real-world medical time-series data is a critical endeavor, essential for early surveillance of disease outbreaks. This paper demonstrates the integration of time-series anomaly detection techniques to develop surveillance systems for disease outbreaks. Utilizing data from Sweden's telephone counseling service (1177), we first illustrate the trends in physical and mental symptoms recorded as contact reasons, offering valuable insights for outbreak detection. Subsequently, an advanced anomaly detection technique is applied incrementally to these time-series symptoms as univariate and multivariate approaches to assess the effectiveness of a machine learning-based method on early detection of the COVID-19 outbreak.

The skåne emergency medicine (SEM) cohort.

BACKGROUND: In the European Union alone, more than 100 million people present to the emergency department (ED) each year, and this has increased steadily year-on-year by 2-3%. Better patient management decisions have the potential to reduce ED crowding, the number of diagnostic tests, the use of inpatient beds, and healthcare costs. METHODS: We have established the Skåne Emergency Medicine (SEM) cohort for developing clinical decision support systems (CDSS) based on artificial intelligence or machine learning as well as traditional statistical methods. The SEM cohort consists of 325 539 unselected unique patients with 630 275 visits from January 1st, 2017 to December 31st, 2018 at eight EDs in the region Skåne in southern Sweden. Data on sociodemographics, previous diseases and current medication are available for each ED patient visit, as well as their chief complaint, test results, disposition and the outcome in the form of subsequent diagnoses, treatments, healthcare costs and mortality within a follow-up period of at least 30 days, and up to 3 years. DISCUSSION: The SEM cohort provides a platform for CDSS research, and we welcome collaboration. In addition, SEM's large amount of real-world patient data with almost complete short-term follow-up will allow research in epidemiology, patient management, diagnostics, prognostics, ED crowding, resource allocation, and social medicine.

A Masked Language Model for Multi-Source EHR Trajectories Contextual Representation Learning.

Using electronic health records data and machine learning to guide future decisions needs to address challenges, including 1) long/short-term dependencies and 2) interactions between diseases and interventions. Bidirectional transformers have effectively addressed the first challenge. Here we tackled the latter challenge by masking one source (e.g., ICD10 codes) and training the transformer to predict it using other sources (e.g., ATC codes).

The implementation of NILS: A web-based artificial neural network decision support tool for noninvasive lymph node staging in breast cancer.

Objective: To implement artificial neural network (ANN) algorithms for noninvasive lymph node staging (NILS) to a decision support tool and facilitate the option to omit surgical axillary staging in breast cancer patients with low-risk of nodal metastasis. Methods: The NILS tool is a further development of an ANN prototype for the prediction of nodal status. Training and internal validation of the original algorithm included 15 clinical and tumor-related variables from a consecutive cohort of 800 breast cancer cases. The updated NILS tool included 10 top-ranked input variables from the original prototype. A workflow with four ANN pathways was additionally developed to allow different combinations of missing preoperative input values. Predictive performances were assessed by area under the receiver operating characteristics curves (AUC) and sensitivity/specificity values at defined cut-points. Clinical utility was presented by estimating possible sentinel lymph node biopsy (SLNB) reduction rates. The principles of user-centered design were applied to develop an interactive web-interface to predict the patient's probability of healthy lymph nodes. A technical validation of the interface was performed using data from 100 test patients selected to cover all combinations of missing histopathological input values. Results: ANN algorithms for the prediction of nodal status have been implemented into the web-based NILS tool for personalized, noninvasive nodal staging in breast cancer. The estimated probability of healthy lymph nodes using the interface showed a complete concordance with estimations from the reference algorithm except in two cases that had been wrongly included (ineligible for the technical validation). NILS predictive performance to distinguish node-negative from node-positive disease, also with missing values, displayed AUC ranged from 0.718 (95% CI, 0.687-0.748) to 0.735 (95% CI, 0.704-0.764), with good calibration. Sensitivity 90% and specificity 34% were demonstrated. The potential to abstain from axillary surgery was observed in 26% of patients using the NILS tool, acknowledging a false negative rate of 10%, which is clinically accepted for the standard SLNB technique. Conclusions: The implementation of NILS into a web-interface are expected to provide the health care with decision support and facilitate preoperative identification of patients who could be good candidates to avoid unnecessary surgical axillary staging.

A machine learning model for prediction of 30-day primary graft failure after heart transplantation.

Background: Primary graft failure (PGF) remains the most common cause of short-term mortality after heart transplantation. The main objective was to develop and validate a risk model for prediction of short-term mortality due to PGF after heart transplantation using the ISHLT Heart Transplant Registry. Methods: We developed a non-linear artificial neural networks (ANN) model to evaluate the association between recipient-donor variables and post-transplant PGF. Patients in the ISHLT registry were randomly divided into derivation and an independent internal validation cohort. The primary endpoint was PGF defined as death within 30 days due to Graft failure or Cardiovascular causes or retransplant within 30 days for causes other than rejection. Results: Among 64,964 adult recipients transplanted between 1994 and 2013, mean age was 51 years and 22% were female. The incidence of PGF up to 30 days was 3.7%. The ANN model selected 33 of 77 risk variables as relevant for PGF prediction. The C-index in the test cohort was 0.70 (95% CI: 0.68-0.71). The risk variables which most influenced the PGF were underlying HF diagnosis, ischemia time and sex, while renal function had a lower influence. Conclusion: An ANN model to predict primary graft dysfunction was derived and independently validated. The good discrimination of the ANN model likely results from its flexibility to model potentially non-linear relationships and interactions. Whether this model with improved discrimination can assist in clinical decisions at the time of transplant should be tested.

Noninvasive Staging of Lymph Node Status in Breast Cancer Using Machine Learning: External Validation and Further Model Development.

BACKGROUND: Most patients diagnosed with breast cancer present with a node-negative disease. Sentinel lymph node biopsy (SLNB) is routinely used for axillary staging, leaving patients with healthy axillary lymph nodes without therapeutic effects but at risk of morbidities from the intervention. Numerous studies have developed nodal status prediction models for noninvasive axillary staging using postoperative data or imaging features that are not part of the diagnostic workup. Lymphovascular invasion (LVI) is a top-ranked predictor of nodal metastasis; however, its preoperative assessment is challenging. OBJECTIVE: This paper aimed to externally validate a multilayer perceptron (MLP) model for noninvasive lymph node staging (NILS) in a large population-based cohort (n=18,633) and develop a new MLP in the same cohort. Data were extracted from the Swedish National Quality Register for Breast Cancer (NKBC, 2014-2017), comprising only routinely and preoperatively available documented clinicopathological variables. A secondary aim was to develop and validate an LVI MLP for imputation of missing LVI status to increase the preoperative feasibility of the original NILS model. METHODS: Three nonoverlapping cohorts were used for model development and validation. A total of 4 MLPs for nodal status and 1 LVI MLP were developed using 11 to 12 routinely available predictors. Three nodal status models were used to account for the different availabilities of LVI status in the cohorts and external validation in NKBC. The fourth nodal status model was developed for 80% (14,906/18,663) of NKBC cases and validated in the remaining 20% (3727/18,663). Three alternatives for imputation of LVI status were compared. The discriminatory capacity was evaluated using the validation area under the receiver operating characteristics curve (AUC) in 3 of the nodal status models. The clinical feasibility of the models was evaluated using calibration and decision curve analyses. RESULTS: External validation of the original NILS model was performed in NKBC (AUC 0.699, 95% CI 0.690-0.708) with good calibration and the potential of sparing 16% of patients with node-negative disease from SLNB. The LVI model was externally validated (AUC 0.747, 95% CI 0.694-0.799) with good calibration but did not improve the discriminatory performance of the nodal status models. A new nodal status model was developed in NKBC without information on LVI (AUC 0.709, 95% CI: 0.688-0.729), with excellent calibration in the holdout internal validation cohort, resulting in the potential omission of 24% of patients from unnecessary SLNBs. CONCLUSIONS: The NILS model was externally validated in NKBC, where the imputation of LVI status did not improve the model's discriminatory performance. A new nodal status model demonstrated the feasibility of using register data comprising only the variables available in the preoperative setting for NILS using machine learning. Future steps include ongoing preoperative validation of the NILS model and extending the model with, for example, mammography images.