Explainable clinical coding with in-domain adapted transformers.

BACKGROUND AND OBJECTIVE: Automatic clinical coding is a crucial task in the process of extracting relevant information from unstructured medical documents contained in Electronic Health Records (EHR). However, most of the existing computer-based methods for clinical coding act as "black boxes", without giving a detailed description of the reasons for the clinical-coding assignments, which greatly limits their applicability to real-world medical scenarios. The objective of this study is to use transformer-based models to effectively tackle explainable clinical-coding. In this way, we require the models to perform the assignments of clinical codes to medical cases, but also to provide the reference in the text that justifies each coding assignment. METHODS: We examine the performance of 3 transformer-based architectures on 3 different explainable clinical-coding tasks. For each transformer, we compare the performance of the original general-domain version with an in-domain version of the model adapted to the specificities of the medical domain. We address the explainable clinical-coding problem as a dual medical named entity recognition (MER) and medical named entity normalization (MEN) task. For this purpose, we have developed two different approaches, namely a multi-task and a hierarchical-task strategy. RESULTS: For each analyzed transformer, the clinical-domain version significantly outperforms the corresponding general domain model across the 3 explainable clinical-coding tasks analyzed in this study. Furthermore, the hierarchical-task approach yields a significantly superior performance than the multi-task strategy. Specifically, the combination of the hierarchical-task strategy with an ensemble approach leveraging the predictive capabilities of the 3 distinct clinical-domain transformers, yields the best obtained results, with f1-score, precision and recall of 0.852, 0.847 and 0.849 on the Cantemist-Norm task and 0.718, 0.566 and 0.633 on the CodiEsp-X task, respectively. CONCLUSIONS: By separately addressing the MER and MEN tasks, as well as by following a context-aware text-classification approach to tackle the MEN task, the hierarchical-task approach effectively reduces the intrinsic complexity of explainable clinical-coding, leading the transformers to establish new SOTA performances for the predictive tasks considered in this study. In addition, the proposed methodology has the potential to be applied to other clinical tasks that require both the recognition and normalization of medical entities.

Machine learning and natural language processing (NLP) approach to predict early progression to first-line treatment in real-world hormone receptor-positive (HR+)/HER2-negative advanced breast cancer patients.

BACKGROUND: CDK4/6 inhibitors plus endocrine therapies are the current standard of care in the first-line treatment of HR+/HER2-negative metastatic breast cancer, but there are no well-established clinical or molecular predictive factors for patient response. In the era of personalised oncology, new approaches for developing predictive models of response are needed. MATERIALS AND METHODS: Data derived from the electronic health records (EHRs) of real-world patients with HR+/HER2-negative advanced breast cancer were used to develop predictive models for early and late progression to first-line treatment. Two machine learning approaches were used: a classic approach using a data set of manually extracted features from reviewed (EHR) patients, and a second approach using natural language processing (NLP) of free-text clinical notes recorded during medical visits. RESULTS: Of the 610 patients included, there were 473 (77.5%) progressions to first-line treatment, of which 126 (20.6%) occurred within the first 6 months. There were 152 patients (24.9%) who showed no disease progression before 28 months from the onset of first-line treatment. The best predictive model for early progression using the manually extracted dataset achieved an area under the curve (AUC) of 0.734 (95% CI 0.687-0.782). Using the NLP free-text processing approach, the best model obtained an AUC of 0.758 (95% CI 0.714-0.800). The best model to predict long responders using manually extracted data obtained an AUC of 0.669 (95% CI 0.608-0.730). With NLP free-text processing, the best model attained an AUC of 0.752 (95% CI 0.705-0.799). CONCLUSIONS: Using machine learning methods, we developed predictive models for early and late progression to first-line treatment of HR+/HER2-negative metastatic breast cancer, also finding that NLP-based machine learning models are slightly better than predictive models based on manually obtained data.

Transfer learning with convolutional neural networks for cancer survival prediction using gene-expression data.

Precision medicine in oncology aims at obtaining data from heterogeneous sources to have a precise estimation of a given patient's state and prognosis. With the purpose of advancing to personalized medicine framework, accurate diagnoses allow prescription of more effective treatments adapted to the specificities of each individual case. In the last years, next-generation sequencing has impelled cancer research by providing physicians with an overwhelming amount of gene-expression data from RNA-seq high-throughput platforms. In this scenario, data mining and machine learning techniques have widely contribute to gene-expression data analysis by supplying computational models to supporting decision-making on real-world data. Nevertheless, existing public gene-expression databases are characterized by the unfavorable imbalance between the huge number of genes (in the order of tenths of thousands) and the small number of samples (in the order of a few hundreds) available. Despite diverse feature selection and extraction strategies have been traditionally applied to surpass derived over-fitting issues, the efficacy of standard machine learning pipelines is far from being satisfactory for the prediction of relevant clinical outcomes like follow-up end-points or patient's survival. Using the public Pan-Cancer dataset, in this study we pre-train convolutional neural network architectures for survival prediction on a subset composed of thousands of gene-expression samples from thirty-one tumor types. The resulting architectures are subsequently fine-tuned to predict lung cancer progression-free interval. The application of convolutional networks to gene-expression data has many limitations, derived from the unstructured nature of these data. In this work we propose a methodology to rearrange RNA-seq data by transforming RNA-seq samples into gene-expression images, from which convolutional networks can extract high-level features. As an additional objective, we investigate whether leveraging the information extracted from other tumor-type samples contributes to the extraction of high-level features that improve lung cancer progression prediction, compared to other machine learning approaches.

Male breast cancer: correlation between immunohistochemical subtyping and PAM50 intrinsic subtypes, and the subsequent clinical outcomes.

Male breast cancer is a rare disease that is still poorly understood. It is mainly classified by immunohistochemistry as a luminal disease. In this study, we assess for the first time the correlation between molecular subtypes based on a validated six-marker immunohistochemical panel and PAM50 signature in male breast cancer, and the subsequent clinical outcome of these different subtypes. We collected 67 surgical specimens of invasive male breast cancer from four different Spanish pathology laboratories. Immunohistochemical staining for the six-marker panel was performed on tissue microarrays. PAM50 subtypes were determined in a research-use-only nCounter Analysis System. We explored the association of immunohistochemical and PAM50 subtypes. Overall survival and disease-free survival were analyzed in the different subtypes of each classification. The distribution of tumor molecular subtypes according PAM50 was: 60% luminal B, 30% luminal A and 10% human epidermal growth factor receptor 2 (Her2) enriched. Only one Her2-enriched tumor was also positive by immunohistochemistry and was treated with trastuzumab. None of the tumors were basal-like. Using immunohistochemical surrogates, 51% of the tumors were luminal B, 44% luminal A, 4% triple-negative and 1% Her2-positive. The clinicopathological characteristics did not differ significantly between immunohistochemical and PAM50 subtypes. We found a significant worse overall survival in Her2-enriched compared with luminal tumors. Male breast cancer seems to be mainly a genomic luminal disease with a predominance of the luminal B subtype. In addition, we found a proportion of patients with Her2-negative by immunohistochemistry but Her2-enriched profile by PAM50 tumors with a worse outcome compared with luminal subtypes that may benefit from anti-Her2 therapies.

Supervised discretization can discover risk groups in cancer survival analysis.

Discretization of continuous variables is a common practice in medical research to identify risk patient groups. This work compares the performance of gold-standard categorization procedures (TNM+A protocol) with that of three supervised discretization methods from Machine Learning (CAIM, ChiM and DTree) in the stratification of patients with breast cancer. The performance for the discretization algorithms was evaluated based on the results obtained after applying standard survival analysis procedures such as Kaplan-Meier curves, Cox regression and predictive modelling. The results show that the application of alternative discretization algorithms could lead the clinicians to get valuable information for the diagnosis and outcome of the disease. Patient data were collected from the Medical Oncology Service of the Hospital Clínico Universitario (Málaga, Spain) considering a follow up period from 1982 to 2008.

Efficient Implementation of the Backpropagation Algorithm in FPGAs and Microcontrollers.

The well-known backpropagation learning algorithm is implemented in a field-programmable gate array (FPGA) board and a microcontroller, focusing in obtaining efficient implementations in terms of a resource usage and computational speed. The algorithm was implemented in both cases using a training/validation/testing scheme in order to avoid overfitting problems. For the case of the FPGA implementation, a new neuron representation that reduces drastically the resource usage was introduced by combining the input and first hidden layer units in a single module. Further, a time-division multiplexing scheme was implemented for carrying out product computations taking advantage of the built-in digital signal processor cores. In both implementations, the floating-point data type representation normally used in a personal computer (PC) has been changed to a more efficient one based on a fixed-point scheme, reducing system memory variable usage and leading to an increase in computation speed. The results show that the modifications proposed produced a clear increase in computation speed in comparison with the standard PC-based implementation, demonstrating the usefulness of the intrinsic parallelism of FPGAs in neurocomputational tasks and the suitability of both implementations of the algorithm for its application to the real world problems.

Application of genetic algorithms and constructive neural networks for the analysis of microarray cancer data.

BACKGROUND: Extracting relevant information from microarray data is a very complex task due to the characteristics of the data sets, as they comprise a large number of features while few samples are generally available. In this sense, feature selection is a very important aspect of the analysis helping in the tasks of identifying relevant genes and also for maximizing predictive information. METHODS: Due to its simplicity and speed, Stepwise Forward Selection (SFS) is a widely used feature selection technique. In this work, we carry a comparative study of SFS and Genetic Algorithms (GA) as general frameworks for the analysis of microarray data with the aim of identifying group of genes with high predictive capability and biological relevance. Six standard and machine learning-based techniques (Linear Discriminant Analysis (LDA), Support Vector Machines (SVM), Naive Bayes (NB), C-MANTEC Constructive Neural Network, K-Nearest Neighbors (kNN) and Multilayer perceptron (MLP)) are used within both frameworks using six free-public datasets for the task of predicting cancer outcome. RESULTS: Better cancer outcome prediction results were obtained using the GA framework noting that this approach, in comparison to the SFS one, leads to a larger selection set, uses a large number of comparison between genetic profiles and thus it is computationally more intensive. Also the GA framework permitted to obtain a set of genes that can be considered to be more biologically relevant. Regarding the different classifiers used standard feedforward neural networks (MLP), LDA and SVM lead to similar and best results, while C-MANTEC and k-NN followed closely but with a lower accuracy. Further, C-MANTEC, MLP and LDA permitted to obtain a more limited set of genes in comparison to SVM, NB and kNN, and in particular C-MANTEC resulted in the most robust classifier in terms of changes in the parameter settings. CONCLUSIONS: This study shows that if prediction accuracy is the objective, the GA-based approach lead to better results respect to the SFS approach, independently of the classifier used. Regarding classifiers, even if C-MANTEC did not achieve the best overall results, the performance was competitive with a very robust behaviour in terms of the parameters of the algorithm, and thus it can be considered as a candidate technique for future studies.

The generalization complexity measure for continuous input data.

We introduce in this work an extension for the generalization complexity measure to continuous input data. The measure, originally defined in Boolean space, quantifies the complexity of data in relationship to the prediction accuracy that can be expected when using a supervised classifier like a neural network, SVM, and so forth. We first extend the original measure for its use with continuous functions to later on, using an approach based on the use of the set of Walsh functions, consider the case of having a finite number of data points (inputs/outputs pairs), that is, usually the practical case. Using a set of trigonometric functions a model that gives a relationship between the size of the hidden layer of a neural network and the complexity is constructed. Finally, we demonstrate the application of the introduced complexity measure, by using the generated model, to the problem of estimating an adequate neural network architecture for real-world data sets.

A microRNA signature associated with early recurrence in breast cancer.

Recurrent breast cancer occurring after the initial treatment is associated with poor outcome. A bimodal relapse pattern after surgery for primary tumor has been described with peaks of early and late recurrence occurring at about 2 and 5 years, respectively. Although several clinical and pathological features have been used to discriminate between low- and high-risk patients, the identification of molecular biomarkers with prognostic value remains an unmet need in the current management of breast cancer. Using microarray-based technology, we have performed a microRNA expression analysis in 71 primary breast tumors from patients that either remained disease-free at 5 years post-surgery (group A) or developed early (group B) or late (group C) recurrence. Unsupervised hierarchical clustering of microRNA expression data segregated tumors in two groups, mainly corresponding to patients with early recurrence and those with no recurrence. Microarray data analysis and RT-qPCR validation led to the identification of a set of 5 microRNAs (the 5-miRNA signature) differentially expressed between these two groups: miR-149, miR-10a, miR-20b, miR-30a-3p and miR-342-5p. All five microRNAs were down-regulated in tumors from patients with early recurrence. We show here that the 5-miRNA signature defines a high-risk group of patients with shorter relapse-free survival and has predictive value to discriminate non-relapsing versus early-relapsing patients (AUC = 0.993, p-value<0.05). Network analysis based on miRNA-target interactions curated by public databases suggests that down-regulation of the 5-miRNA signature in the subset of early-relapsing tumors would result in an overall increased proliferative and angiogenic capacity. In summary, we have identified a set of recurrence-related microRNAs with potential prognostic value to identify patients who will likely develop metastasis early after primary breast surgery.

Differential outcome of concurrent radiotherapy plus epidermal growth factor receptor inhibitors versus radiotherapy plus cisplatin in patients with human papillomavirus-related head and neck cancer.

BACKGROUND: Human papillomavirus (HPV)-related head and neck cancer has been associated with an improved prognosis in patients treated with radiotherapy (RT) +/- chemotherapy (CT); however, RT combined with epidermal growth factor receptor (EGFR) inhibitors has not been fully studied in this group of patients. METHODS: Immunohistochemical expression of p16 and PCR of HPV16 DNA were retrospectively analyzed in tumor blocks from 108 stage III/IV head and neck cancer patients treated with RT+CT (56) or RT+EGFR inhibitors (52). Disease-free survival (DFS) and overall survival (OS) were analyzed by the Kaplan-Meier method. RESULTS: DNA of HPV16 was found in 12 of 108 tumors (11%) and p16 positivity in 18 tumors (17%), with similar rates in both arms of treatment. After a median follow-up time of 35 months (range 6-135), p16-positive patients treated with RT+EGFR inhibitors showed improved survival compared with those treated with RT+CT (2-year OS 88% vs. 60%, HR 0.18; 95% CI 0.04 to 0.88; p = 0.01; and 2-year DFS 75% vs. 47%, HR 0.17; 95% CI 0.03 to 0.8; p = 0.01). However, no differences were observed in p16-negative patients (2-year OS 56% vs. 53%, HR 0.97; 95% CI 0.55 to 1.7; p = 0.9; and 2-year DFS 43% vs. 45%, HR 0.99; 95% CI 0.57 to 1.7; p = 0.9). CONCLUSIONS: This is the first study to show that p16-positive patients may benefit more from RT+EGFR inhibitors than conventional RT+CT. These results are hypothesis-generating and should be confirmed in prospective trials.

Serum protein levels following surgery in breast cancer patients: a protein microarray approach.

Surgery is the primary treatment for non-metastatic breast cancer. However, the risk of early recurrence remains after surgical removal of the primary tumor. Recurrence is suggested to result from hidden micrometastatic foci, which are triggered to escape from dormancy by surgical resection of the primary tumor. In this study, we focused on the differential impact of breast surgery on the serum profiles of early breast cancer patients and healthy women. Serum samples from invasive breast cancer patients, in situ carcinoma breast cancer patients and healthy women were analyzed using reverse phase protein array technology. Samples were collected prior to breast surgery and 24 h following breast surgery. Both the expression level and the velocity of 42 serum proteins were quantified and compared among groups. We found that surgery increased the concentration of several proteins (CSF1, THSB2, IL6, IL7, IL16, FasL and VEGF-B) in the overall population. Compared with healthy women and patients with non-invasive tumors, invasive tumor patients exhibited higher preoperative levels of several serum proteins, such as αFP, IFNß1, VEGF-A, IL18, E-cadherin or CD31, and lower postoperative levels of TNFα and IL5. Similarly, we detected significant surgery-induced changes in the velocity of VEGF-A and IL16 accumulation in samples derived from invasive breast cancer patients. In conclusion, breast surgery induced distinct changes in the concentrations and dynamics of serum proteins in invasive breast cancer patients compared with healthy women and non-invasive tumor patients.

C-Mantec: a novel constructive neural network algorithm incorporating competition between neurons.

C-Mantec is a novel neural network constructive algorithm that combines competition between neurons with a stable modified perceptron learning rule. The neuron learning is governed by the thermal perceptron rule that ensures stability of the acquired knowledge while the architecture grows and while the neurons compete for new incoming information. Competition makes it possible that even after new units have been added to the network, existing neurons still can learn if the incoming information is similar to their stored knowledge, and this constitutes a major difference with existing constructing algorithms. The new algorithm is tested on two different sets of benchmark problems: a Boolean function set used in logic circuit design and a well studied set of real world problems. Both sets were used to analyze the size of the constructed architectures and the generalization ability obtained and to compare the results with those from other standard and well known classification algorithms. The problem of overfitting is also analyzed, and a new built-in method to avoid its effects is devised and successfully applied within an active learning paradigm that filter noisy examples. The results show that the new algorithm generates very compact neural architectures with state-of-the-art generalization capabilities.

Missing data imputation using statistical and machine learning methods in a real breast cancer problem.

OBJECTIVES: Missing data imputation is an important task in cases where it is crucial to use all available data and not discard records with missing values. This work evaluates the performance of several statistical and machine learning imputation methods that were used to predict recurrence in patients in an extensive real breast cancer data set. MATERIALS AND METHODS: Imputation methods based on statistical techniques, e.g., mean, hot-deck and multiple imputation, and machine learning techniques, e.g., multi-layer perceptron (MLP), self-organisation maps (SOM) and k-nearest neighbour (KNN), were applied to data collected through the "El Álamo-I" project, and the results were then compared to those obtained from the listwise deletion (LD) imputation method. The database includes demographic, therapeutic and recurrence-survival information from 3679 women with operable invasive breast cancer diagnosed in 32 different hospitals belonging to the Spanish Breast Cancer Research Group (GEICAM). The accuracies of predictions on early cancer relapse were measured using artificial neural networks (ANNs), in which different ANNs were estimated using the data sets with imputed missing values. RESULTS: The imputation methods based on machine learning algorithms outperformed imputation statistical methods in the prediction of patient outcome. Friedman's test revealed a significant difference (p=0.0091) in the observed area under the ROC curve (AUC) values, and the pairwise comparison test showed that the AUCs for MLP, KNN and SOM were significantly higher (p=0.0053, p=0.0048 and p=0.0071, respectively) than the AUC from the LD-based prognosis model. CONCLUSION: The methods based on machine learning techniques were the most suited for the imputation of missing values and led to a significant enhancement of prognosis accuracy compared to imputation methods based on statistical procedures.

Neuronal selectivity, population sparseness, and ergodicity in the inferior temporal visual cortex.

The sparseness of the encoding of stimuli by single neurons and by populations of neurons is fundamental to understanding the efficiency and capacity of representations in the brain, and was addressed as follows. The selectivity and sparseness of firing to visual stimuli of single neurons in the primate inferior temporal visual cortex were measured to a set of 20 visual stimuli including objects and faces in macaques performing a visual fixation task. Neurons were analysed with significantly different responses to the stimuli. The firing rate distribution of 36% of the neurons was exponential. Twenty-nine percent of the neurons had too few low rates to be fitted by an exponential distribution, and were fitted by a gamma distribution. Interestingly, the raw firing rate distribution taken across all neurons fitted an exponential distribution very closely. The sparseness a (s) or selectivity of the representation of the set of 20 stimuli provided by each of these neurons (which takes a maximal value of 1.0) had an average across all neurons of 0.77, indicating a rather distributed representation. The sparseness of the representation of a given stimulus by the whole population of neurons, the population sparseness a (p), also had an average value of 0.77. The similarity of the average single neuron selectivity a (s) and population sparseness for any one stimulus taken at any one time a (p) shows that the representation is weakly ergodic. For this to occur, the different neurons must have uncorrelated tuning profiles to the set of stimuli.

Information in the first spike, the order of spikes, and the number of spikes provided by neurons in the inferior temporal visual cortex.

Information theoretic analyses showed that for single inferior temporal neurons and neuronal populations, more information was encoded in 20 or more ms by all the spikes available than just by the first spike in the same time window about which of 20 objects or faces was shown. Further, the temporal order in which the first spike arrived from different simultaneously recorded neurons did not encode more information than was present in the first spike or the spike counts. Thus information transmission in the inferior temporal cortex by the number of spikes in even short time windows is fast, and provides more information than only the first spike, or the spike order from different neurons.

Informed prognosis [corrected] after abdominal aortic aneurysm repair using predictive modeling techniques [corrected].

OBJECTIVE: To identify the best method for the prediction of postoperative mortality in individual abdominal aortic aneurysm surgery (AAA) patients by comparing statistical modelling with artificial neural networks' (ANN) and clinicians' estimates. METHODS: An observational multicenter study was conducted of prospectively collected postoperative Acute Physiology and Chronic Health Evaluation II data for a 9-year period from 24 intensive care units (ICU) in the Thames region of the United Kingdom. The study cohort consisted of 1205 elective and 546 emergency AAA patients. Four independent physiologic variables-age, acute physiology score, emergency operation, and chronic health evaluation-were used to develop multiple regression and ANN models to predict in-hospital mortality. The models were developed on 75% of the patient population and their validity tested on the remaining 25%. The results from these two models were compared with the observed outcome and clinicians' estimates by using measures of calibration, discrimination, and subgroup analysis. RESULTS: Observed in-hospital mortality for elective surgery was 9.3% (95% confidence interval [CI], 7.7% to 11.1%) and for emergency surgery, 46.7% (95% CI, 42.5 to 51.0%). The ANN and the statistical models were both more accurate than the clinicians' predictions. Only the statistical model was internally valid, however, when applied to the validation set of observations, as evidenced by calibration (Hosmer-Lemeshow C statistic, 14.97; P = .060), discrimination properties (area under receiver operating characteristic curve, 0.869; 95% CI, 0.824 to 0.913), and subgroup analysis. CONCLUSIONS: The prediction of in-hospital mortality in AAA patients by multiple regression is more accurate than clinicians' estimates or ANN modelling. Clinicians can use this statistical model as an objective adjunct to generate informed prognosis.