A weighted distance-based dynamic ensemble regression framework for gastric cancer survival time prediction.

Accurate prediction of gastric cancer patient survival time is essential for clinical decision-making. However, unified static models lack specificity and flexibility in predictions owing to the varying survival outcomes among gastric cancer patients. We address these problems by using an ensemble learning approach and adaptively assigning greater weights to similar patients to make more targeted predictions when predicting an individual's survival time. We treat these problems as regression problems and introduce a weighted dynamic ensemble regression framework. To better identify similar patients, we devise a method to measure patient similarity, considering the diverse impacts of features. Subsequently, we use this measure to design both a weighted K-means clustering method and a fuzzy K-means sampling technique to group patients and train corresponding base regressors. To achieve more targeted predictions, we calculate the weight of each base regressor based on the similarity between the patient to be predicted and the patient clusters, culminating in the integration of the results. The model is validated on a dataset of 7791 patients, outperforming other models in terms of three evaluation metrics, namely, the root mean square error, mean absolute error, and the coefficient of determination. The weighted dynamic ensemble regression strategy can improve the baseline model by 1.75%, 2.12%, and 13.45% in terms of the three respective metrics while also mitigating the imbalanced survival time distribution issue. This enhanced performance has been statistically validated, even when tested on six public datasets with different sizes. By considering feature variations, patients with distinct survival profiles can be effectively differentiated, and the model predictive performance can be enhanced. The results generated by our proposed model can be invaluable in guiding decisions related to treatment plans and resource allocation. Furthermore, the model has the potential for broader applications in prognosis for other types of cancers or similar regression problems in various domains.

Enhancing urban flood forecasting in drainage systems using dynamic ensemble-based data mining.

This study presents a novel approach for urban flood forecasting in drainage systems using a dynamic ensemble-based data mining model which has yet to be utilised properly in this context. The proposed method incorporates an event identification technique and rainfall feature extraction to develop weak learner data mining models. These models are then stacked to create a time-series ensemble model using a decision tree algorithm and confusion matrix-based blending method. The proposed model was compared to other commonly used ensemble models in a real-world urban drainage system in the UK. The results show that the proposed model achieves a higher hit rate compared to other benchmark models, with a hit rate of around 85% vs 70 % for the next 3 h of forecasting. Additionally, the proposed smart model can accurately classify various timesteps of flood or non-flood events without significant lag times, resulting in fewer false alarms, reduced unnecessary risk management actions, and lower costs in real-time early warning applications. The findings also demonstrate that two features, "antecedent precipitation history" and "seasonal time occurrence of rainfall," significantly enhance the accuracy of flood forecasting with a hit rate accuracy ranging from 60 % to 10 % for a lead time of 15 min to 3 h.

Multi-source dynamic ensemble prediction of infectious disease and application in COVID-19 case.

Background: The development of an epidemic always exhibits multiwave oscillation owing to various anthropogenic sources of transmission. Particularly in populated areas, the large-scaled human mobility led to the transmission of the virus faster and more complex. The accurate prediction of the spread of infectious diseases remains a problem. To solve this problem, we propose a new method called the multi-source dynamic ensemble prediction (MDEP) method that incorporates a modified susceptible-exposed-infected-removed (SEIR) model to improve the accuracy of the prediction result. Methods: The modified SEIR model is based on the compartment model, which is suitable for local-scale and confined spaces, where human mobility on a large scale is not considered. Moreover, compartmental models cannot be used to predict multiwave epidemics. The proposed MDEP method can remedy defects in the compartment model. In this study, multi-source prediction was made on the development of coronavirus disease 2019 (COVID-19) and dynamically assembled to obtain the final integrated result. We used the real epidemic data of COVID-19 in three cities in China: Beijing, Lanzhou, and Beihai. Epidemiological data were collected from 17 April, 2022 to 12 August, 2022. Results: Compared to the one-wave modified SEIR model, the MDEP method can depict the multiwave development of COVID-19. The MDEP method was applied to predict the number of cumulative cases of recent COVID-19 outbreaks in the aforementioned cities in China. The average accuracy rates in Beijing, Lanzhou, and Beihai were 89.15%, 91.74%, and 94.97%, respectively. Conclusions: The MDEP method improved the prediction accuracy of COVID-19. With further application to other infectious diseases, the MDEP method will provide accurate predictions of infectious diseases and aid governments make appropriate directives.

Prediction of posttraumatic functional recovery in middle-aged and older patients through dynamic ensemble selection modeling.

Introduction: Age-specific risk factors may delay posttraumatic functional recovery; complex interactions exist between these factors. In this study, we investigated the prediction ability of machine learning models for posttraumatic (6 months) functional recovery in middle-aged and older patients on the basis of their preexisting health conditions. Methods: Data obtained from injured patients aged ≥45 years were divided into training-validation (n = 368) and test (n = 159) data sets. The input features were the sociodemographic characteristics and baseline health conditions of the patients. The output feature was functional status 6 months after injury; this was assessed using the Barthel Index (BI). On the basis of their BI scores, the patients were categorized into functionally independent (BI >60) and functionally dependent (BI ≤60) groups. The permutation feature importance method was used for feature selection. Six algorithms were validated through cross-validation with hyperparameter optimization. The algorithms exhibiting satisfactory performance were subjected to bagging to construct stacking, voting, and dynamic ensemble selection models. The best model was evaluated on the test data set. Partial dependence (PD) and individual conditional expectation (ICE) plots were created. Results: In total, nineteen of twenty-seven features were selected. Logistic regression, linear discrimination analysis, and Gaussian Naive Bayes algorithms exhibited satisfactory performances and were, therefore, used to construct ensemble models. The k-Nearest Oracle Elimination model outperformed the other models when evaluated on the training-validation data set (sensitivity: 0.732, 95% CI: 0.702-0.761; specificity: 0.813, 95% CI: 0.805-0.822); it exhibited compatible performance on the test data set (sensitivity: 0.779, 95% CI: 0.559-0.950; specificity: 0.859, 95% CI: 0.799-0.912). The PD and ICE plots showed consistent patterns with practical tendencies. Conclusion: Preexisting health conditions can predict long-term functional outcomes in injured middle-aged and older patients, thus predicting prognosis and facilitating clinical decision-making.

A classification algorithm based on dynamic ensemble selection to predict mutational patterns of the envelope protein in HIV-infected patients.

BACKGROUND: Therapeutics against the envelope (Env) proteins of human immunodeficiency virus type 1 (HIV-1) effectively reduce viral loads in patients. However, due to mutations, new therapy-resistant Env variants frequently emerge. The sites of mutations on Env that appear in each patient are considered random and unpredictable. Here we developed an algorithm to estimate for each patient the mutational state of each position based on the mutational state of adjacent positions on the three-dimensional structure of the protein. METHODS: We developed a dynamic ensemble selection algorithm designated k-best classifiers. It identifies the best classifiers within the neighborhood of a new observation and applies them to predict the variability state of each observation. To evaluate the algorithm, we applied amino acid sequences of Envs from 300 HIV-1-infected individuals (at least six sequences per patient). For each patient, amino acid variability values at all Env positions were mapped onto the three-dimensional structure of the protein. Then, the variability state of each position was estimated by the variability at adjacent positions of the protein. RESULTS: The proposed algorithm showed higher performance than the base learner and a panel of classification algorithms. The mutational state of positions in the high-mannose patch and CD4-binding site of Env, which are targeted by multiple therapeutics, was predicted well. Importantly, the algorithm outperformed other classification techniques for predicting the variability state at multi-position footprints of therapeutics on Env. CONCLUSIONS: The proposed algorithm applies a dynamic classifier-scoring approach that increases its performance relative to other classification methods. Better understanding of the spatiotemporal patterns of variability across Env may lead to new treatment strategies that are tailored to the unique mutational patterns of each patient. More generally, we propose the algorithm as a new high-performance dynamic ensemble selection technique.

Reliable Identification and Interpretation of Single-Cell Molecular Heterogeneity and Transcriptional Regulation using Dynamic Ensemble Pruning.

Unsupervised clustering is an essential step in identifying cell types from single-cell RNA sequencing (scRNA-seq) data. However, a common issue with unsupervised clustering models is that the optimization direction of the objective function and the final generated clustering labels in the absence of supervised information may be inconsistent or even arbitrary. To address this challenge, a dynamic ensemble pruning framework (DEPF) is proposed to identify and interpret single-cell molecular heterogeneity. In particular, a silhouette coefficient-based indicator is developed to determine the optimization direction of the bi-objective function. In addition, a hierarchical autoencoder is employed to project the high-dimensional data onto multiple low-dimensional latent space sets, and then a clustering ensemble is produced in the latent space by the basic clustering algorithm. Following that, a bi-objective fruit fly optimization algorithm is designed to prune dynamically the low-quality basic clustering in the ensemble. Multiple experiments are conducted on 28 real scRNA-seq datasets and one large real scRNA-seq dataset from diverse platforms and species to validate the effectiveness of the DEPF. In addition, biological interpretability and transcriptional and post-transcriptional regulatory are conducted to explore biological patterns from the cell types identified, which could provide novel insights into characterizing the mechanisms.

DMEformer: A newly designed dynamic model ensemble transformer for crude oil futures prediction.

Crude oil futures prediction plays an important role in ensuring sustainable energy development. However, the performance of existing models is not satisfactory, which limits its further application. The poor performance mainly results from the lack of data mining of economic models and the poor stability of most data analysis models. To solve the above problems, this paper proposes a new dynamic model ensemble transformer (DMEformer). The model uses three different Transformer variants as base models. It not only ensures the difference of base models but also makes the prediction results of base models not to appear disparity. In addition, NSGA-II is adopted to ensemble the results of base models, which considers both the modeling stability and accuracy in the optimization. Finally, the proposed model adopts a dynamic ensemble scheme, which could timely adjust the weight vector according to the fluctuation of energy futures. It further improves the reliability of the model. Comparative experiments from the perspective of single models and ensemble models are also designed. The following conclusions can be drawn from the experimental results: (1) The proposed dynamic ensemble method can improve the performance of the base model and traditional static ensemble method by 16% and 5% respectively. (2) DMEformer can achieve better performance than 20 other models, and its accuracy and MAPE values were 72.5% and 2.8043%, respectively. (3) The proposed model can accurately predict crude oil futures, which provides effective support for energy regulation and sustainable development.

Multilayer dynamic ensemble model for intensive care unit mortality prediction of neonate patients.

Robust and rabid mortality prediction is crucial in intensive care units because it is considered one of the critical steps for treating patients with serious conditions. Combining mortality prediction with the length of stay (LoS) prediction adds another level of importance to these models. No studies in the literature predict such tasks for neonates, especially using time-series data and dynamic ensemble techniques. Dynamic ensembles are novel techniques that dynamically select the base classifiers for each new case. Medically, implementing an accurate machine learning model is insufficient to gain the trust of physicians. The model must be able to justify its decisions. While explainable AI (XAI) techniques can be used to handle this challenge, no studies have been done in this regard for neonate monitoring in the neonatal intensive care unit (NICU). This study utilizes advanced machine learning approaches to predict mortality and LoS through data-driven learning. We propose a multilayer dynamic ensemble-based model to predict mortality as a classification task and LoS as a regression task for neonates admitted to the NICU. The model has been built based on the patient's time-series data of the first 24 h in the NICU. We utilized a cohort of 3,133 infants from the MIMIC-III real dataset to build and optimize the selected algorithms. It has shown that the dynamic ensemble models achieved better results than other classifiers, and static ensemble regressors achieved better results than classical machine learning regressors. The proposed optimized model is supported by three well-known explainability techniques of SHAP, decision tree visualization, and rule-based system. To provide online assistance to physicians in monitoring and managing neonates in the NICU, we implemented a web-based clinical decision support system based on the most accurate models and selected XAI techniques. The code of the proposed models is publicly available at https://github.com/InfoLab-SKKU/neonateMortalityPrediction.

Segmentation technique and dynamic ensemble selection to enhance glaucoma severity detection.

The severity of glaucoma can be observed by categorising glaucoma diseases into several classes based on a classification process. The two most suitable parameters, cup-to-disc ratio (CDR) and peripapillary atrophy (PPA), which are commonly used to identify glaucoma are utilized in this study to strengthen the classification. First, an active contour snake (ACS) is employed to retrieve both optic disc (OD) and optic cup (OC) values, which are required to calculate the CDR. Moreover, Otsu segmentation and thresholding techniques are used to identify PPA, and the features are then extracted using a grey-level co-occurrence matrix (GLCM). An advanced segmentation technique, combined with an improved classifier called dynamic ensemble selection (DES), is proposed to classify glaucoma. Because DES is generally used to handle an imbalanced dataset, the proposed model is expected to detect glaucoma severity and determine the subsequent treatment accurately. The proposed model obtains a higher mean accuracy (0.96) than the deep learning-based U-Net (0.90) when evaluated using three datasets of 250 retinal fundus images (200 training, 50 testings) based on the 5-fold cross-validation scheme.

A novel combined dynamic ensemble selection model for imbalanced data to detect COVID-19 from complete blood count.

BACKGROUND: As blood testing is radiation-free, low-cost and simple to operate, some researchers use machine learning to detect COVID-19 from blood test data. However, few studies take into consideration the imbalanced data distribution, which can impair the performance of a classifier. METHOD: A novel combined dynamic ensemble selection (DES) method is proposed for imbalanced data to detect COVID-19 from complete blood count. This method combines data preprocessing and improved DES. Firstly, we use the hybrid synthetic minority over-sampling technique and edited nearest neighbor (SMOTE-ENN) to balance data and remove noise. Secondly, in order to improve the performance of DES, a novel hybrid multiple clustering and bagging classifier generation (HMCBCG) method is proposed to reinforce the diversity and local regional competence of candidate classifiers. RESULTS: The experimental results based on three popular DES methods show that the performance of HMCBCG is better than only use bagging. HMCBCG+KNE obtains the best performance for COVID-19 screening with 99.81% accuracy, 99.86% F1, 99.78% G-mean and 99.81% AUC. CONCLUSION: Compared to other advanced methods, our combined DES model can improve accuracy, G-mean, F1 and AUC of COVID-19 screening.

3E-Net: Entropy-Based Elastic Ensemble of Deep Convolutional Neural Networks for Grading of Invasive Breast Carcinoma Histopathological Microscopic Images.

Automated grading systems using deep convolution neural networks (DCNNs) have proven their capability and potential to distinguish between different breast cancer grades using digitized histopathological images. In digital breast pathology, it is vital to measure how confident a DCNN is in grading using a machine-confidence metric, especially with the presence of major computer vision challenging problems such as the high visual variability of the images. Such a quantitative metric can be employed not only to improve the robustness of automated systems, but also to assist medical professionals in identifying complex cases. In this paper, we propose Entropy-based Elastic Ensemble of DCNN models (3E-Net) for grading invasive breast carcinoma microscopy images which provides an initial stage of explainability (using an uncertainty-aware mechanism adopting entropy). Our proposed model has been designed in a way to (1) exclude images that are less sensitive and highly uncertain to our ensemble model and (2) dynamically grade the non-excluded images using the certain models in the ensemble architecture. We evaluated two variations of 3E-Net on an invasive breast carcinoma dataset and we achieved grading accuracy of 96.15% and 99.50%.

A novel dynamic multi-criteria ensemble selection mechanism applied to drinking water quality anomaly detection.

The provision of clean and safe drinking water is a crucial task for water supply companies from all over the world. To this end, automatic anomaly detection plays a critical role in drinking water quality monitoring. Recent anomaly detection studies use techniques that focus on a single global objective. Yet, companies need solutions that better balance the trade-off between false positives (FPs), which lead to financial losses to water companies, and false negatives (FNs), which severely impact public health and damage the environment. This work proposes a novel dynamic multi-criteria ensemble selection mechanism to cope with both problems simultaneously: the non-dominated local class-specific accuracy (NLCA). Moreover, experiments rely on recent time series related classification metrics to assess the predictive performance. Results on data from a real-world water distribution system show that NLCA outperforms other ensemble learning and dynamic ensemble selection techniques by more than 15% in terms of time series related F1 scores. As a conclusion, NLCA enables the development of stronger anomaly detection systems for drinking water quality monitoring. The proposed technique also offers a new perspective on dynamic ensemble selection, which can be applied to different classification tasks to balance conflicting criteria.

Differentiation of pain levels by deploying various electroencephalogram synchronization features and a dynamic ensemble selection mechanism.

Objective: The aim of this study was to measure pain intensity in an objective manner by analyzing electroencephalogram (EEG) signals. Although this problem has attracted the attention of researchers, increasing the resolution of this measurement by increasing the number of pain states significantly decreases the accuracy of pain level classification.Approach: To overcome this drawback, we adopt state-of-the-art synchronization schemes to measure the linear, nonlinear and generalized synchronization between different EEG channels. Thirty-two subjects executed the cold pressor task and experienced five defined levels of pain while their EEGs were recorded. Due to the large number of synchronization features from 34 channels, the most discriminative features were selected using the greedy overall relevancy method. The selected features were applied to a dynamic ensemble selection system.Main results: Our experiment provides 85.6% accuracy over the five classes, which significantly improves upon the results of past research. Moreover, we observed that the selected features belong to the channels placed over the ridge of the cortex, the area responsible for processing somatic sensation arising from nociceptive temperature. As expected, we noted that continuation of the painful stimulus for minutes engaged regions beyond the sensorimotor cortex (e.g. the prefrontal cortex).Significance: We conclude that the amount of synchronization between scalp EEG channels is an informative tool in revealing the pain sensation.

Bearing Fault Diagnosis with a Feature Fusion Method Based on an Ensemble Convolutional Neural Network and Deep Neural Network.

Rolling bearings are the core components of rotating machinery. Their health directly affects the performance, stability and life of rotating machinery. To prevent possible damage, it is necessary to detect the condition of rolling bearings for fault diagnosis. With the rapid development of intelligent fault diagnosis technology, various deep learning methods have been applied in fault diagnosis in recent years. Convolution neural networks (CNN) have shown high performance in feature extraction. However, the pooling operation of CNN can lead to the loss of much valuable information and the relationship between the whole and the part may be ignored. In this study, we proposed CNNEPDNN, a novel bearing fault diagnosis model based on ensemble deep neural network (DNN) and CNN. We firstly trained CNNEPDNN model. Each of its local networks was trained with different training datasets. The CNN used vibration sensor signals as the input, whereas the DNN used nine time-domain statistical features from bearing vibration sensor signals as the input. Each local network of CNNEPDNN extracted different features from its own trained dataset, thus we fused features with different discrimination for fault recognition. CNNEPDNN was tested under 10 fault conditions based on the bearing data from Bearing Data Center of Case Western Reserve University (CWRU). To evaluate the proposed model, four aspects were analyzed: convergence speed of training loss function, test accuracy, F-Score and the feature clustering result by t-distributed stochastic neighbor embedding (t-SNE) visualization. The training loss function of the proposed model converged more quickly than the local models under different loads. The test accuracy of the proposed model is better than that of CNN, DNN and BPNN. The F-Score value of the model is higher than that of CNN model, and the feature clustering effect of the proposed model was better than that of CNN.

Dynamic ensemble selection of learner-descriptor classifiers to assess curve types in adolescent idiopathic scoliosis.

While classification is important for assessing adolescent idiopathic scoliosis (AIS), it however suffers from low interobserver and intraobserver reliability. Classification using ensemble methods may contribute to improving reliability using the proper 2D and 3D images of spine curvature features. In this study, we present two new techniques to describe the spine, namely, leave-one-out and fan leave-one-out. Using these techniques, three descriptors are computed from a stereoradiographic 3D reconstruction to describe the relationship between a vertebra and its neighbors. A dynamic ensemble selection method is introduced for automatic spine classification. The performance of the method is evaluated on a dataset containing 962 3D spine models categorized according to three curve types. With a log loss of 0.5623, the dynamic ensemble selection outperforms voting and stacking ensemble learning techniques. This method can improve intraobserver and interobserver reliability, identify the best combination of descriptors for characterizing spine curve types, and provide assistance to clinicians in the form of information to classify borderline curvature types. Graphical abstract ᅟ.

Cooperative strategy for a dynamic ensemble of classification models in clinical applications: the case of MRI vertebral compression fractures.

PURPOSE: In clinical practice, the constructive consultation among experts improves the reliability of the diagnosis and leads to the definition of the treatment plan for the patient. Aggregation of the different opinions collected by many experts can be performed at the level of patient information, abnormality delineation, or final assessment. METHODS: In this study, we present a novel cooperative strategy that exploits the dynamic contribution of the classification models composing the ensemble to make the final class assignment. As a proof of concept, we applied the proposed approach to the assessment of malignant infiltration in 103 vertebral compression fractures in magnetic resonance images. RESULTS: The results obtained with repeated random subsampling and receiver operating characteristic analysis indicate that the cooperative system statistically improved ([Formula: see text]) the classification accuracy of individual modules as well as of that based on the manual segmentation of the fractures provided by the experts. CONCLUSIONS: The performances have been also compared with those obtained with those of standard ensemble classification algorithms showing superior results.

Parallel gene selection and dynamic ensemble pruning based on Affinity Propagation.

Gene selection and sample classification based on gene expression data are important research areas in bioinformatics. Selecting important genes closely related to classification is a challenging task due to high dimensionality and small sample size of microarray data. Extended rough set based on neighborhood has been successfully applied to gene selection, as it can select attributes without redundancy and deal with numerical attributes directly. However, the computation of approximations in rough set is extremely time consuming. In this paper, in order to accelerate the process of gene selection, a parallel computation method is proposed to calculate approximations of intersection neighborhood rough set. Furthermore, a novel dynamic ensemble pruning approach based on Affinity Propagation clustering and dynamic pruning framework is proposed to reduce memory usage and computational cost. Experimental results on three Arabidopsis thaliana biotic and abiotic stress response datasets demonstrate that the proposed method can obtain better classification performance than ensemble method with gene pre-selection.