Deep-learning and data-resampling: A novel approach to predict cyanobacterial alert levels in a reservoir.

The proliferation of harmful algal blooms results in adverse impacts on aquatic ecosystems and public health. Early warning system monitors algal bloom occurrences and provides management strategies for promptly addressing high-concentration algal blooms following their occurrence. In this study, we aimed to develop a proactive prediction model for cyanobacterial alert levels to enable efficient decision-making in management practices. We utilized 11 years of water quality, hydrodynamic, and meteorological data from a reservoir that experiences frequent harmful cyanobacterial blooms in summer. We used these data to construct a deep-learning model, specifically a 1D convolution neural network (1D-CNN) model, to predict cyanobacterial alert levels one week in advance. However, the collected distribution of algal alert levels was imbalanced, leading to the biased training of data-driven models and performance degradation in model predictions. Therefore, an adaptive synthetic sampling method was applied to address the imbalance in the minority class data and improve the predictive performance of the 1D-CNN. The adaptive synthetic sampling method resolved the imbalance in the data during the training phase by incorporating an additional 156 and 196 data points for the caution and warning levels, respectively. The selected optimal 1D-CNN model with a filter size of 5 and comprising 16 filters achieved training and testing prediction accuracies of 97.3% and 85.0%, respectively. During the test phase, the prediction accuracies for each algal alert level (L-0, L-1, and L-2) were 89.9%, 79.2%, and 71.4%, respectively, indicating reasonably consistent predictive results for all three alert levels. Therefore, the use of synthetic data addressed data imbalances and enhanced the predictive performance of the data-driven model. The reliable forecasts produced by the improved model can support the development of management strategies to mitigate harmful algal blooms in reservoirs and can aid in building an early warning system to facilitate effective responses.

Comparative Analysis of a Locally Resampling MR Elastography Reconstruction Algorithm in Liver Fibrosis.

BACKGROUND: In magnetic resonance elastography (MRE), the precision of the observed mechanical depends on the ratio between mechanical wavelength and spatial resolution. Since the mechanical wavelength may vary with actuation frequency, between patients and depending on position, a unique spatial resolution may not always generate an optimal ratio for multifrequency acquisitions, in patients with varying degrees of disease or in mechanically heterogeneous organs. PURPOSE: To describe an MRE reconstruction algorithm that adjusts the ratio between shear wavelength and pixel size, by locally resampling the matrix of shear displacement, and to assess its performance relative to existing reconstructions in different use cases. STUDY TYPE: Prospective. POPULATION: Four phantoms, 20 healthy volunteers (5 men, median age 34, range 20-56) and 46 patients with nonalcoholic fatty liver disease (37 men, median age 63, range 33-83). FIELD STRENGTH/SEQUENCE: A 3 T; gradient-echo elastography sequence with 40 Hz, 60 Hz, and 80 Hz frequencies. ASSESSMENT: For each algorithm, phantoms stiffness were compared against their nominal values, repeatability was calculated in healthy volunteers, and diagnostic performance in detecting advanced liver fibrosis was assessed in 46 patients. STATISTICAL TESTS: Linear regression was used to evaluate the agreement between stiffness values and phantoms stiffnesses. Bland-Altman method was used to evaluate repeatability in volunteers. The ability to diagnose advanced fibrosis was assessed by receiver operating curve analysis (with Youden index thresholds). Significance was considered at P value of 0.05. RESULTS: From the linear regression, the slope closest to 1 is provided by MARS (40 Hz) and k-MDEV (60H, 80 Hz). Repeatability index was best with MDEV (23%) and lowest with k-MDEV (53%). The best performance in detecting advanced fibrosis was provided by MARS at 40 Hz (area under the operating curve, AUC = 0.88), k-MDEV and MARS at 60 Hz (AUC = 0.91), and multimodel direct inversion (MMDI) and MARS at 80 Hz (AUC = 0.90). DATA CONCLUSION: MARS shows the best diagnostic performance to detect advanced fibrosis and the second-best results in phantoms after k-MDEV. EVIDENCE LEVEL: 1. TECHNICAL EFFICACY: Stage 2.

Confidence intervals of the difference between areas under two ROC curves in matched-pair experiments.

In a matched-pair study, when outcomes of two diagnostic tests are ordinal/continuous, the difference between two correlated areas under ROC curves (AUCs) is usually used to compare the overall discriminatory ability of two diagnostic tests. This article considers confidence interval (CI) construction problems of difference between two correlated AUCs in a matched-pair experiment, and proposes 13 hybrid CIs based on variance estimates recovery with the maximum likelihood estimation, Delong's statistic, Wilson score statistic (WS) and WS with continuity correction, the modified Wald statistic (MW) and MW with continuity correction and Agresti-Coull statistic, and three Bootstrap-resampling-based CIs. For comparison, we present traditional parametric and nonparametric CIs. Simulation studies are conducted to assess the performance of the proposed CIs in terms of empirical coverage probabilities, empirical interval widths, and ratios of the mesial noncoverage probabilities to the noncoverage probabilities. Two examples from clinical studies are illustrated by the proposed methodologies. Empirical results evidence that the hybrid Agresti-Coull CI with the empirical estimation (EAC) behaved most satisfactorily because its coverage probability was quite close to the prespecified confidence level with short interval width. Hence, we recommend the usage of the EAC CI in applications.

Simultaneous confidence interval for assessing non-inferiority with assay sensitivity in a three-arm trial with binary endpoints.

A three-arm trial including an experimental treatment, an active reference treatment and a placebo is often used to assess the non-inferiority (NI) with assay sensitivity of an experimental treatment. Various hypothesis-test-based approaches via a fraction or pre-specified margin have been proposed to assess the NI with assay sensitivity in a three-arm trial. There is little work done on confidence interval in a three-arm trial. This paper develops a hybrid approach to construct simultaneous confidence interval for assessing NI and assay sensitivity in a three-arm trial. For comparison, we present normal-approximation-based and bootstrap-resampling-based simultaneous confidence intervals. Simulation studies evidence that the hybrid approach with the Wilson score statistic performs better than other approaches in terms of empirical coverage probability and mesial-non-coverage probability. An example is used to illustrate the proposed approaches.

Construction of confidence intervals for the risk differences in stratified design with correlated bilateral data.

A stratified study is often designed for adjusting a confounding effect or effect of different centers/groups in two treatments or diagnostic tests, and the risk difference is one of the most frequently used indices in comparing efficiency between two treatments or diagnostic tests. This article presented five simultaneous confidence intervals (CIs) for risk differences in stratified bilateral designs accounting for the intraclass correlation and developed seven CIs for the common risk difference under the homogeneity assumption. The performance of the CIs is evaluated with respect to the empirical coverage probabilities, empirical coverage widths and ratios of mesial noncoverage probability and the noncoverage probability under various scenarios. Empirical results show that Wald simultaneous CI, Haldane simultaneous CI, Score simultaneous CI based on Bonferroni method and simultaneous CI based on bootstrap-resampling method perform satisfactorily and hence be recommended for applications, the CI based on the weighted-least-square (WLS) estimator, the CIs based on Mantel-Haenszel estimator, the CI based on Cochran statistic and the CI based on Score statistic for the common risk difference behave well even under small sample sizes. A real data example is used to demonstrate the proposed methodologies.

A flexible and coherent test/estimation procedure based on restricted mean survival times for censored time-to-event data in randomized clinical trials.

In randomized clinical trials where time-to-event is the primary outcome, almost routinely, the logrank test is prespecified as the primary test and the hazard ratio is used to quantify treatment effect. If the ratio of 2 hazard functions is not constant, the logrank test is not optimal and the interpretation of hazard ratio is not obvious. When such a nonproportional hazards case is expected at the design stage, the conventional practice is to prespecify another member of weighted logrank tests, eg, Peto-Prentice-Wilcoxon test. Alternatively, one may specify a robust test as the primary test, which can capture various patterns of difference between 2 event time distributions. However, most of those tests do not have companion procedures to quantify the treatment difference, and investigators have fallen back on reporting treatment effect estimates not associated with the primary test. Such incoherence in the "test/estimation" procedure may potentially mislead clinicians/patients who have to balance risk-benefit for treatment decision. To address this, we propose a flexible and coherent test/estimation procedure based on restricted mean survival time, where the truncation time τ is selected data dependently. The proposed procedure is composed of a prespecified test and an estimation of corresponding robust and interpretable quantitative treatment effect. The utility of the new procedure is demonstrated by numerical studies based on 2 randomized cancer clinical trials; the test is dramatically more powerful than the logrank, Wilcoxon tests, and the restricted mean survival time-based test with a fixed τ, for the patterns of difference seen in these cancer clinical trials.

Analysis of small sample size studies using nonparametric bootstrap test with pooled resampling method.

Experimental studies in biomedical research frequently pose analytical problems related to small sample size. In such studies, there are conflicting findings regarding the choice of parametric and nonparametric analysis, especially with non-normal data. In such instances, some methodologists questioned the validity of parametric tests and suggested nonparametric tests. In contrast, other methodologists found nonparametric tests to be too conservative and less powerful and thus preferred using parametric tests. Some researchers have recommended using a bootstrap test; however, this method also has small sample size limitation. We used a pooled method in nonparametric bootstrap test that may overcome the problem related with small samples in hypothesis testing. The present study compared nonparametric bootstrap test with pooled resampling method corresponding to parametric, nonparametric, and permutation tests through extensive simulations under various conditions and using real data examples. The nonparametric pooled bootstrap t-test provided equal or greater power for comparing two means as compared with unpaired t-test, Welch t-test, Wilcoxon rank sum test, and permutation test while maintaining type I error probability for any conditions except for Cauchy and extreme variable lognormal distributions. In such cases, we suggest using an exact Wilcoxon rank sum test. Nonparametric bootstrap paired t-test also provided better performance than other alternatives. Nonparametric bootstrap test provided benefit over exact Kruskal-Wallis test. We suggest using nonparametric bootstrap test with pooled resampling method for comparing paired or unpaired means and for validating the one way analysis of variance test results for non-normal data in small sample size studies. Copyright © 2017 John Wiley & Sons, Ltd.

Predicting the restricted mean event time with the subject's baseline covariates in survival analysis.

For designing, monitoring, and analyzing a longitudinal study with an event time as the outcome variable, the restricted mean event time (RMET) is an easily interpretable, clinically meaningful summary of the survival function in the presence of censoring. The RMET is the average of all potential event times measured up to a time point τ and can be estimated consistently by the area under the Kaplan-Meier curve over $[0, \tau ]$. In this paper, we study a class of regression models, which directly relates the RMET to its "baseline" covariates for predicting the future subjects' RMETs. Since the standard Cox and the accelerated failure time models can also be used for estimating such RMETs, we utilize a cross-validation procedure to select the "best" among all the working models considered in the model building and evaluation process. Lastly, we draw inferences for the predicted RMETs to assess the performance of the final selected model using an independent data set or a "hold-out" sample from the original data set. All the proposals are illustrated with the data from the an HIV clinical trial conducted by the AIDS Clinical Trials Group and the primary biliary cirrhosis study conducted by the Mayo Clinic.

A versatile test for equality of two survival functions based on weighted differences of Kaplan-Meier curves.

With censored event time observations, the logrank test is the most popular tool for testing the equality of two underlying survival distributions. Although this test is asymptotically distribution free, it may not be powerful when the proportional hazards assumption is violated. Various other novel testing procedures have been proposed, which generally are derived by assuming a class of specific alternative hypotheses with respect to the hazard functions. The test considered by Pepe and Fleming (1989) is based on a linear combination of weighted differences of the two Kaplan-Meier curves over time and is a natural tool to assess the difference of two survival functions directly. In this article, we take a similar approach but choose weights that are proportional to the observed standardized difference of the estimated survival curves at each time point. The new proposal automatically makes weighting adjustments empirically. The new test statistic is aimed at a one-sided general alternative hypothesis and is distributed with a short right tail under the null hypothesis but with a heavy tail under the alternative. The results from extensive numerical studies demonstrate that the new procedure performs well under various general alternatives with a caution of a minor inflation of the type I error rate when the sample size is small or the number of observed events is small. The survival data from a recent cancer comparative study are utilized for illustrating the implementation of the process.

Testing goodness-of-fit for the proportional hazards model based on nested case-control data.

Nested case-control sampling is a popular design for large epidemiological cohort studies due to its cost effectiveness. A number of methods have been developed for the estimation of the proportional hazards model with nested case-control data; however, the evaluation of modeling assumption is less attended. In this article, we propose a class of goodness-of-fit test statistics for testing the proportional hazards assumption based on nested case-control data. The test statistics are constructed based on asymptotically mean-zero processes derived from Samuelsen's maximum pseudo-likelihood estimation method. In addition, we develop an innovative resampling scheme to approximate the asymptotic distribution of the test statistics while accounting for the dependent sampling scheme of nested case-control design. Numerical studies are conducted to evaluate the performance of our proposed approach, and an application to the Wilms' Tumor Study is given to illustrate the methodology.

Model Checking Techniques for Assessing Functional Form Specifications in Censored Linear Regression Models.

In this paper we develop model checking techniques for assessing functional form specifications of covariates in censored linear regression models. These procedures are based on a censored data analog to taking cumulative sums of "robust" residuals over the space of the covariate under investigation. These cumulative sums are formed by integrating certain Kaplan-Meier estimators and may be viewed as "robust" censored data analogs to the processes considered by Lin, Wei & Ying (2002). The null distributions of these stochastic processes can be approximated by the distributions of certain zero-mean Gaussian processes whose realizations can be generated by computer simulation. Each observed process can then be graphically compared with a few realizations from the Gaussian process. We also develop formal test statistics for numerical comparison. Such comparisons enable one to assess objectively whether an apparent trend seen in a residual plot reects model misspecification or natural variation. We illustrate the methods with a well known dataset. In addition, we examine the finite sample performance of the proposed test statistics in simulation experiments. In our simulation experiments, the proposed test statistics have good power of detecting misspecification while at the same time controlling the size of the test.

Two-Stage Adaptive Design for Prognostic Biomarker Signatures with a Survival Endpoint.

Cancer biomarker discoveries typically involve utilizing patient specimens. In practice, there is often strong desire to preserve high quality biospecimens for studies that are most likely to yield useful information. Previously, we proposed a two-stage adaptive design for binary endpoints which terminates the biomarker study in a futility interim if the model performance is unsatisfactory. In this work, we extend the two-stage design framework to accommodate time-to-event endpoints. The first stage of the procedure involves testing whether the measure of discrimination for survival models (C-index) exceeds a pre-specified threshold. We describe the computation of cross-validated C-index and evaluation of the statistical significance using re-sampling techniques. The second stage involves an independent model validation. Our simulation studies show that under the null hypothesis, the proposed design maintains type I error at the nominal level and has high probabilities of terminating the study early. Under the alternative hypothesis, power of the design is a function of the true event proportion, the sample size, and the targeted improvement in the discriminant measure. We apply the method to design of a prognostic biomarker study in patients with triple-negative breast cancer. Some practical aspects of the proposed method are discussed.

A Resampling Method to Improve the Prognostic Model of End-Stage Kidney Disease: A Better Strategy for Imbalanced Data.

Background: Prognostic models can help to identify patients at risk for end-stage kidney disease (ESKD) at an earlier stage to provide preventive medical interventions. Previous studies mostly applied the Cox proportional hazards model. The aim of this study is to present a resampling method, which can deal with imbalanced data structure for the prognostic model and help to improve predictive performance. Methods: The electronic health records of patients with chronic kidney disease (CKD) older than 50 years during 2005-2015 collected from primary care in Belgium were used (n = 11,645). Both the Cox proportional hazards model and the logistic regression analysis were applied as reference model. Then, the resampling method, the Synthetic Minority Over-Sampling Technique-Edited Nearest Neighbor (SMOTE-ENN), was applied as a preprocessing procedure followed by the logistic regression analysis. The performance was evaluated by accuracy, the area under the curve (AUC), confusion matrix, and F 3 score. Results: The C statistics for the Cox proportional hazards model was 0.807, while the AUC for the logistic regression analysis was 0.700, both on a comparable level to previous studies. With the model trained on the resampled set, 86.3% of patients with ESKD were correctly identified, although it was at the cost of the high misclassification rate of negative cases. The F 3 score was 0.245, much higher than 0.043 for the logistic regression analysis and 0.022 for the Cox proportional hazards model. Conclusion: This study pointed out the imbalanced data structure and its effects on prediction accuracy, which were not thoroughly discussed in previous studies. We were able to identify patients with high risk for ESKD better from a clinical perspective by using the resampling method. But, it has the limitation of the high misclassification of negative cases. The technique can be widely used in other clinical topics when imbalanced data structure should be considered.

Confidence intervals for assessing equivalence of two treatments with combined unilateral and bilateral data.

Responses from the paired organs are generally highly correlated in bilateral studies, statistical procedures ignoring the correlation could lead to incorrect results. Note the intraclass correlation in the study of combined unilateral and bilateral outcomes; 11 confidence intervals (CIs) including 7 asymptotic CIs and 4 Bootstrap-resampling CIs for assessing the equivalence of 2 treatments are derived under Rosner's correlated binary data model. Performance is evaluated with respect to the empirical coverage probability (ECP), the empirical coverage width (ECW) and the ratio of the mesial non-coverage probability to the non-coverage probability (RMNCP) via simulation studies. Simulation results show that (i) all CIs except for the Wald CI and the bias-corrected Bootstrap percentile CI generally produce satisfactory ECPs and hence are recommended; (ii) all CIs except for the bias-corrected Bootstrap percentile CI provide preferred RMNCPs and are more symmetrical; (iii) as the measurement of the dependence increases, the ECWs of all CIs except for the score CI and the profile likelihood CI show increasing patterns that look like linear, while there is no obvious pattern on the ECPs of all CIs except for the profile likelihood CI. A data set from an otolaryngologic study is used to illustrate the proposed methods.

Simultaneous Inference of Treatment Effect Modification by Intermediate Response Endpoint Principal Strata with Application to Vaccine Trials.

In randomized clinical trials, researchers are often interested in identifying an inexpensive intermediate study endpoint (typically a biomarker) that is a strong effect modifier of the treatment effect on a longer-term clinical endpoint of interest. Motivated by randomized placebo-controlled preventive vaccine efficacy trials, within the principal stratification framework a pseudo-score type estimator has been proposed to estimate disease risks conditional on the counter-factual biomarker of interest under each treatment assignment to vaccine or placebo, yielding an estimator of biomarker conditional vaccine efficacy. This method can be used for trial designs that use baseline predictors of the biomarker and/or designs that vaccinate disease-free placebo recipients at the end of the trial. In this article, we utilize the pseudo-score estimator to estimate the biomarker conditional vaccine efficacy adjusting for baseline covariates. We also propose a perturbation resampling method for making simultaneous inference on conditional vaccine efficacy over the values of the biomarker. We illustrate our method with datasets from two phase 3 dengue vaccine efficacy trials.

Key feature selection and risk prediction for lane-changing behaviors based on vehicles' trajectory data.

Risky lane-changing (LC) behavior of vehicles on the road has negative effects on traffic safety. This study presents a research framework for key feature selection and risk prediction of car's LC behavior on the highway based on vehicles' trajectory dataset. To the best of our knowledge, this is the first study that focuses on key feature selection and risk prediction for LC behavior on the highway. From the vehicles' trajectory dataset, we extract car's candidate features and apply fault tree analysis and k-Means clustering algorithm to determine the LC risk level based on the performance indicator of Crash Potential Index (CPI). Random Forest (RF) classifier is applied to select key features from car's candidate features and predict LC risk level. This study also proposes a method to evaluate the resampling methods to resample the LC risk dataset in terms of fitness performance and prediction performance. The cars' trajectory data collected from the Next Generation Simulation (NGSIM) dataset is used for framework development and verification. The sensitivity analysis of CPI indicates that the following cars in the original lane and target lane are respectively the safest and riskiest cars of the surrounding cars in an LC event. The results of resampling method evaluation show that SMOTETomek, which is less likely to be overfitting and has high prediction performance, is well suited for resampling the LC risk dataset on which RF classifier is trained. The results of key feature selection imply that the individual behaviors of the LC car and its surrounding cars in the original lane, the interactions between the LC car and its surrounding cars, and the interactions between the surrounding cars in the target lane (especially the interaction of the cars' accelerations) are of importance to the LC risk.

Cost-Effective Extreme Case-Control Design Using a Resampling Method.

Nested case-control sampling design is a popular method in a cohort study whose events are often rare. The controls are randomly selected with or without the matching variable fully observed across all cohort samples to control confounding factors. In this article, we propose a new nested case-control sampling design incorporating both extreme case-control design and a resampling technique. This new algorithm has two main advantages with respect to the conventional nested case-control design. First, it inherits the strength of extreme case-control design such that it does not require the risk sets in each event time to be specified. Second, the target number of controls can only be determined by the budget and time constraints and the resampling method allows an under sampling design, which means that the total number of sampled controls can be smaller than the number of cases. A simulation study demonstrated that the proposed algorithm performs well even when we have a smaller number of controls compared with the number of cases. The proposed sampling algorithm is applied to a public data collected for "Thorotrast Study."

Bootstrap resampling approach to disaggregate analysis of road crashes in Hong Kong.

Road safety affects health and development worldwide; thus, it is essential to examine the factors that influence crashes and injuries. As the relationships between crashes, crash severity, and possible risk factors can vary depending on the type of collision, we attempt to develop separate prediction models for different crash types (i.e., single- versus multi-vehicle crashes and slight injury versus killed and serious injury crashes). Taking advantage of the availability of crash and traffic data disaggregated by time and space, it is possible to identify the factors that may contribute to crash risks in Hong Kong, including traffic flow, road design, and weather conditions. To remove the effects of excess zeros on prediction performance in a highly disaggregated crash prediction model, a bootstrap resampling method is applied. The results indicate that more accurate and reliable parameter estimates, with reduced standard errors, can be obtained with the use of a bootstrap resampling method. Results revealed that factors including rainfall, geometric design, traffic control, and temporal variations all determined the crash risk and crash severity. This helps to shed light on the development of remedial engineering and traffic management and control measures.

Resampling methods in Microsoft Excel® for estimating reference intervals.

Computer-intensive resampling/bootstrap methods are feasible when calculating reference intervals from non-Gaussian or small reference samples. Microsoft Excel® in version 2010 or later includes natural functions, which lend themselves well to this purpose including recommended interpolation procedures for estimating 2.5 and 97.5 percentiles. The purpose of this paper is to introduce the reader to resampling estimation techniques in general and in using Microsoft Excel® 2010 for the purpose of estimating reference intervals in particular. Parametric methods are preferable to resampling methods when the distributions of observations in the reference samples is Gaussian or can transformed to that distribution even when the number of reference samples is less than 120. Resampling methods are appropriate when the distribution of data from the reference samples is non-Gaussian and in case the number of reference individuals and corresponding samples are in the order of 40. At least 500-1000 random samples with replacement should be taken from the results of measurement of the reference samples.

Seasonal habitat preference by the flagship species Testudo hermanni: Implications for the conservation of coastal dunes.

In this study, we explored if, how, and when the European Union habitats (EU sensu Habitats Directive 92/43/CEE) are used by the flagship species Testudo hermanni in a well-preserved coastal dune system of the Italian peninsula. Radio telemetry data and fine-scale vegetation habitat mapping were used to address the following questions: (a) is each EU habitat used differentially by Hermann's tortoises? (b) is there any seasonal variation in this utilization pattern? (c) how does each habitat contribute to the ecological requirements of the tortoises? Nine tortoises were fitted with transmitters and monitored for the entire season of activity. The eight EU habitats present in the study area were surveyed and mapped using GIS. The seasonal preferential use or avoidance of each habitat was tested by comparing, through bootstrap tests, the proportion of habitat occupied (piTh) with the proportion of available habitat in the entire landscape (piL). The analysis of 340 spatial locations showed a marked preference for the Cisto-Lavanduletalia dune sclerophyllous scrubs (EU code 2260) and a seasonal selection of Juniperus macrocarpa bushes (EU code 2250(*)), wooded dunes with Pinus (EU code 2270) and mosaic of dune grasslands and sclerophyllous scrubs (EU codes 2230, 2240, 2260). Seasonal variation of habitat preference was interpreted in light of the different feeding, thermoregulation and reproductive needs of the tortoises. Our results stress the ecological value of EU coastal dune habitats and suggest prioritization of conservation efforts in these ecosystems.