An open challenge to advance probabilistic forecasting for dengue epidemics.

A wide range of research has promised new tools for forecasting infectious disease dynamics, but little of that research is currently being applied in practice, because tools do not address key public health needs, do not produce probabilistic forecasts, have not been evaluated on external data, or do not provide sufficient forecast skill to be useful. We developed an open collaborative forecasting challenge to assess probabilistic forecasts for seasonal epidemics of dengue, a major global public health problem. Sixteen teams used a variety of methods and data to generate forecasts for 3 epidemiological targets (peak incidence, the week of the peak, and total incidence) over 8 dengue seasons in Iquitos, Peru and San Juan, Puerto Rico. Forecast skill was highly variable across teams and targets. While numerous forecasts showed high skill for midseason situational awareness, early season skill was low, and skill was generally lowest for high incidence seasons, those for which forecasts would be most valuable. A comparison of modeling approaches revealed that average forecast skill was lower for models including biologically meaningful data and mechanisms and that both multimodel and multiteam ensemble forecasts consistently outperformed individual model forecasts. Leveraging these insights, data, and the forecasting framework will be critical to improve forecast skill and the application of forecasts in real time for epidemic preparedness and response. Moreover, key components of this project-integration with public health needs, a common forecasting framework, shared and standardized data, and open participation-can help advance infectious disease forecasting beyond dengue.

Predicting influenza with dynamical methods.

BACKGROUND: Prediction of influenza weeks in advance can be a useful tool in the management of cases and in the early recognition of pandemic influenza seasons. METHODS: This study explores the prediction of influenza-like-illness incidence using both epidemiological and climate data. It uses Lorenz's well-known Method of Analogues, but with two novel improvements. Firstly, it determines internal parameters using the implicit near-neighbor distances in the data, and secondly, it employs climate data (mean dew point) to screen analogue near-neighbors and capture the hidden dynamics of disease spread. RESULTS: These improvements result in the ability to forecast, four weeks in advance, the total number of cases and the incidence at the peak with increased accuracy. In most locations the total number of cases per year and the incidence at the peak are forecast with less than 15 % root-mean-square (RMS) Error, and in some locations with less than 10 % RMS Error. CONCLUSIONS: The use of additional variables that contribute to the dynamics of influenza spread can greatly improve prediction accuracy.

Data-driven approach for creating synthetic electronic medical records.

BACKGROUND: New algorithms for disease outbreak detection are being developed to take advantage of full electronic medical records (EMRs) that contain a wealth of patient information. However, due to privacy concerns, even anonymized EMRs cannot be shared among researchers, resulting in great difficulty in comparing the effectiveness of these algorithms. To bridge the gap between novel bio-surveillance algorithms operating on full EMRs and the lack of non-identifiable EMR data, a method for generating complete and synthetic EMRs was developed. METHODS: This paper describes a novel methodology for generating complete synthetic EMRs both for an outbreak illness of interest (tularemia) and for background records. The method developed has three major steps: 1) synthetic patient identity and basic information generation; 2) identification of care patterns that the synthetic patients would receive based on the information present in real EMR data for similar health problems; 3) adaptation of these care patterns to the synthetic patient population. RESULTS: We generated EMRs, including visit records, clinical activity, laboratory orders/results and radiology orders/results for 203 synthetic tularemia outbreak patients. Validation of the records by a medical expert revealed problems in 19% of the records; these were subsequently corrected. We also generated background EMRs for over 3000 patients in the 4-11 yr age group. Validation of those records by a medical expert revealed problems in fewer than 3% of these background patient EMRs and the errors were subsequently rectified. CONCLUSIONS: A data-driven method was developed for generating fully synthetic EMRs. The method is general and can be applied to any data set that has similar data elements (such as laboratory and radiology orders and results, clinical activity, prescription orders). The pilot synthetic outbreak records were for tularemia but our approach may be adapted to other infectious diseases. The pilot synthetic background records were in the 4-11 year old age group. The adaptations that must be made to the algorithms to produce synthetic background EMRs for other age groups are indicated.

Evaluation of sliding baseline methods for spatial estimation for cluster detection in the biosurveillance system.

BACKGROUND: The Centers for Disease Control and Prevention's (CDC's) BioSense system provides near-real time situational awareness for public health monitoring through analysis of electronic health data. Determination of anomalous spatial and temporal disease clusters is a crucial part of the daily disease monitoring task. Our study focused on finding useful anomalies at manageable alert rates according to available BioSense data history. METHODS: The study dataset included more than 3 years of daily counts of military outpatient clinic visits for respiratory and rash syndrome groupings. We applied four spatial estimation methods in implementations of space-time scan statistics cross-checked in Matlab and C. We compared the utility of these methods according to the resultant background cluster rate (a false alarm surrogate) and sensitivity to injected cluster signals. The comparison runs used a spatial resolution based on the facility zip code in the patient record and a finer resolution based on the residence zip code. RESULTS: Simple estimation methods that account for day-of-week (DOW) data patterns yielded a clear advantage both in background cluster rate and in signal sensitivity. A 28-day baseline gave the most robust results for this estimation; the preferred baseline is long enough to remove daily fluctuations but short enough to reflect recent disease trends and data representation. Background cluster rates were lower for the rash syndrome counts than for the respiratory counts, likely because of seasonality and the large scale of the respiratory counts. CONCLUSION: The spatial estimation method should be chosen according to characteristics of the selected data streams. In this dataset with strong day-of-week effects, the overall best detection performance was achieved using subregion averages over a 28-day baseline stratified by weekday or weekend/holiday behavior. Changing the estimation method for particular scenarios involving different spatial resolution or other syndromes can yield further improvement.

Ensemble method for dengue prediction.

BACKGROUND: In the 2015 NOAA Dengue Challenge, participants made three dengue target predictions for two locations (Iquitos, Peru, and San Juan, Puerto Rico) during four dengue seasons: 1) peak height (i.e., maximum weekly number of cases during a transmission season; 2) peak week (i.e., week in which the maximum weekly number of cases occurred); and 3) total number of cases reported during a transmission season. A dengue transmission season is the 12-month period commencing with the location-specific, historical week with the lowest number of cases. At the beginning of the Dengue Challenge, participants were provided with the same input data for developing the models, with the prediction testing data provided at a later date. METHODS: Our approach used ensemble models created by combining three disparate types of component models: 1) two-dimensional Method of Analogues models incorporating both dengue and climate data; 2) additive seasonal Holt-Winters models with and without wavelet smoothing; and 3) simple historical models. Of the individual component models created, those with the best performance on the prior four years of data were incorporated into the ensemble models. There were separate ensembles for predicting each of the three targets at each of the two locations. PRINCIPAL FINDINGS: Our ensemble models scored higher for peak height and total dengue case counts reported in a transmission season for Iquitos than all other models submitted to the Dengue Challenge. However, the ensemble models did not do nearly as well when predicting the peak week. CONCLUSIONS: The Dengue Challenge organizers scored the dengue predictions of the Challenge participant groups. Our ensemble approach was the best in predicting the total number of dengue cases reported for transmission season and peak height for Iquitos, Peru.

Prediction of Peaks of Seasonal Influenza in Military Health-Care Data.

Influenza is a highly contagious disease that causes seasonal epidemics with significant morbidity and mortality. The ability to predict influenza peak several weeks in advance would allow for timely preventive public health planning and interventions to be used to mitigate these outbreaks. Because influenza may also impact the operational readiness of active duty personnel, the US military places a high priority on surveillance and preparedness for seasonal outbreaks. A method for creating models for predicting peak influenza visits per total health-care visits (ie, activity) weeks in advance has been developed using advanced data mining techniques on disparate epidemiological and environmental data. The model results are presented and compared with those of other popular data mining classifiers. By rigorously testing the model on data not used in its development, it is shown that this technique can predict the week of highest influenza activity for a specific region with overall better accuracy than other methods examined in this article.

Assessment of damage in an eight-oscillator circuit using dynamical forcing.

We employ chaotic interrogation of a circuit simulation of a structure in order to test for damage to the structure. The circuit simulation provides a realistic test of our attractor-based method and permits close control over parameters in the structure. In this circuit, simulating an eight-degree-of-freedom spring-mass system, we were able to detect changes of as little as 2% in the coupling between two oscillators in the circuit. This corresponded to detection of a 2% loss in stiffness to one spring in the modeled system.

Construction and validation of synthetic electronic medical records.

There is a current and pressing need for a test bed of electronic medical records (EMRs) to insure consistent development, validation and verification of public health related algorithms that operate on EMRs. However, access to full EMRs is limited and not generally available to the academic algorithm developers who support the public health community. This paper describes a set of algorithms that produce synthetic EMRs using real EMRs as a model. The algorithms were used to generate a pilot set of over 3000 synthetic EMRs that are currently available on CDC's Public Health grid. The properties of the synthetic EMRs were validated, both in the entire aggregate data set and for individual (synthetic) patients. We describe how the algorithms can be extended to produce records beyond the initial pilot data set.

A unified approach to attractor reconstruction.

In the analysis of complex, nonlinear time series, scientists in a variety of disciplines have relied on a time delayed embedding of their data, i.e., attractor reconstruction. The process has focused primarily on intuitive, heuristic, and empirical arguments for selection of the key embedding parameters, delay and embedding dimension. This approach has left several longstanding, but common problems unresolved in which the standard approaches produce inferior results or give no guidance at all. We view the current reconstruction process as unnecessarily broken into separate problems. We propose an alternative approach that views the problem of choosing all embedding parameters as being one and the same problem addressable using a single statistical test formulated directly from the reconstruction theorems. This allows for varying time delays appropriate to the data and simultaneously helps decide on embedding dimension. A second new statistic, undersampling, acts as a check against overly long time delays and overly large embedding dimension. Our approach is more flexible than those currently used, but is more directly connected with the mathematical requirements of embedding. In addition, the statistics developed guide the user by allowing optimization and warning when embedding parameters are chosen beyond what the data can support. We demonstrate our approach on uni- and multivariate data, data possessing multiple time scales, and chaotic data. This unified approach resolves all the main issues in attractor reconstruction.

Using chaotic forcing to detect damage in a structure.

In this work we develop a numerical test for Holder continuity and apply it and another test for continuity to the difficult problem of detecting damage in structures. We subject a thin metal plate with incremental damage to the plate changes, its filtering properties, and therefore the phase space trajectories of the response chaotic excitation of various bandwidths. Damage to the plate changes its filtering properties and therefore the phase space of the response. Because the data are multivariate (the plate is instrumented with multiple sensors) we use a singular value decomposition of the set of the output time series to reduce the embedding dimension of the response time series. We use two geometric tests to compare an attractor reconstructed from data from an undamaged structure to that reconstructed from data from a damaged structure. These two tests translate to testing for both generalized and differentiable synchronization between responses. We show loss of synchronization of responses with damage to the structure.