An open challenge to advance probabilistic forecasting for dengue epidemics.

A wide range of research has promised new tools for forecasting infectious disease dynamics, but little of that research is currently being applied in practice, because tools do not address key public health needs, do not produce probabilistic forecasts, have not been evaluated on external data, or do not provide sufficient forecast skill to be useful. We developed an open collaborative forecasting challenge to assess probabilistic forecasts for seasonal epidemics of dengue, a major global public health problem. Sixteen teams used a variety of methods and data to generate forecasts for 3 epidemiological targets (peak incidence, the week of the peak, and total incidence) over 8 dengue seasons in Iquitos, Peru and San Juan, Puerto Rico. Forecast skill was highly variable across teams and targets. While numerous forecasts showed high skill for midseason situational awareness, early season skill was low, and skill was generally lowest for high incidence seasons, those for which forecasts would be most valuable. A comparison of modeling approaches revealed that average forecast skill was lower for models including biologically meaningful data and mechanisms and that both multimodel and multiteam ensemble forecasts consistently outperformed individual model forecasts. Leveraging these insights, data, and the forecasting framework will be critical to improve forecast skill and the application of forecasts in real time for epidemic preparedness and response. Moreover, key components of this project-integration with public health needs, a common forecasting framework, shared and standardized data, and open participation-can help advance infectious disease forecasting beyond dengue.

Ensemble method for dengue prediction.

BACKGROUND: In the 2015 NOAA Dengue Challenge, participants made three dengue target predictions for two locations (Iquitos, Peru, and San Juan, Puerto Rico) during four dengue seasons: 1) peak height (i.e., maximum weekly number of cases during a transmission season; 2) peak week (i.e., week in which the maximum weekly number of cases occurred); and 3) total number of cases reported during a transmission season. A dengue transmission season is the 12-month period commencing with the location-specific, historical week with the lowest number of cases. At the beginning of the Dengue Challenge, participants were provided with the same input data for developing the models, with the prediction testing data provided at a later date. METHODS: Our approach used ensemble models created by combining three disparate types of component models: 1) two-dimensional Method of Analogues models incorporating both dengue and climate data; 2) additive seasonal Holt-Winters models with and without wavelet smoothing; and 3) simple historical models. Of the individual component models created, those with the best performance on the prior four years of data were incorporated into the ensemble models. There were separate ensembles for predicting each of the three targets at each of the two locations. PRINCIPAL FINDINGS: Our ensemble models scored higher for peak height and total dengue case counts reported in a transmission season for Iquitos than all other models submitted to the Dengue Challenge. However, the ensemble models did not do nearly as well when predicting the peak week. CONCLUSIONS: The Dengue Challenge organizers scored the dengue predictions of the Challenge participant groups. Our ensemble approach was the best in predicting the total number of dengue cases reported for transmission season and peak height for Iquitos, Peru.

Prediction of Peaks of Seasonal Influenza in Military Health-Care Data.

Influenza is a highly contagious disease that causes seasonal epidemics with significant morbidity and mortality. The ability to predict influenza peak several weeks in advance would allow for timely preventive public health planning and interventions to be used to mitigate these outbreaks. Because influenza may also impact the operational readiness of active duty personnel, the US military places a high priority on surveillance and preparedness for seasonal outbreaks. A method for creating models for predicting peak influenza visits per total health-care visits (ie, activity) weeks in advance has been developed using advanced data mining techniques on disparate epidemiological and environmental data. The model results are presented and compared with those of other popular data mining classifiers. By rigorously testing the model on data not used in its development, it is shown that this technique can predict the week of highest influenza activity for a specific region with overall better accuracy than other methods examined in this article.

Fuzzy association rule mining and classification for the prediction of malaria in South Korea.

BACKGROUND: Malaria is the world's most prevalent vector-borne disease. Accurate prediction of malaria outbreaks may lead to public health interventions that mitigate disease morbidity and mortality. METHODS: We describe an application of a method for creating prediction models utilizing Fuzzy Association Rule Mining to extract relationships between epidemiological, meteorological, climatic, and socio-economic data from Korea. These relationships are in the form of rules, from which the best set of rules is automatically chosen and forms a classifier. Two classifiers have been built and their results fused to become a malaria prediction model. Future malaria cases are predicted as Low, Medium or High, where these classes are defined as a total of 0-2, 3-16, and above 17 cases, respectively, for a region in South Korea during a two-week period. Based on user recommendations, HIGH is considered an outbreak. RESULTS: Model accuracy is described by Positive Predictive Value (PPV), Sensitivity, and F-score for each class, computed on test data not previously used to develop the model. For predictions made 7-8 weeks in advance, model PPV and Sensitivity are 0.842 and 0.681, respectively, for the HIGH classes. The F0.5 and F3 scores (which combine PPV and Sensitivity) are 0.804 and 0.694, respectively, for the HIGH classes. The overall FARM results (as measured by F-scores) are significantly better than those obtained by Decision Tree, Random Forest, Support Vector Machine, and Holt-Winters methods for the HIGH class. For the Medium class, Random Forest and FARM obtain comparable results, with FARM being better at F0.5, and Random Forest obtaining a higher F3. CONCLUSIONS: A previously described method for creating disease prediction models has been modified and extended to build models for predicting malaria. In addition, some new input variables were used, including indicators of intervention measures. The South Korea malaria prediction models predict Low, Medium or High cases 7-8 weeks in the future. This paper demonstrates that our data driven approach can be used for the prediction of different diseases.

Preliminary development of a fiber optic sensor for measuring bilirubin.

Preliminary development of a fiber optic bilirubin sensor is described, where an unclad sensing portion is used to provide evanescent wave interaction of the transmitted light with the chemical environment. By using a wavelength corresponding to a bilirubin absorption peak, the Beer-Lambert Law can be used to relate the concentration of bilirubin surrounding the sensing portion to the amount of absorbed light. Initial testing in vitro suggests that the sensor response is consistent with the results of bulk absorption measurements as well as the Beer-Lambert Law. In addition, it is found that conjugated and unconjugated bilirubin have different peak absorption wavelengths, so that two optical frequencies may potentially be used to measure both types of bilirubin. Future development of this device could provide a means of real-time, point-of-care monitoring of intravenous bilirubin in critical care neonates with hyperbilirubinemia.

Prediction of high incidence of dengue in the Philippines.

BACKGROUND: Accurate prediction of dengue incidence levels weeks in advance of an outbreak may reduce the morbidity and mortality associated with this neglected disease. Therefore, models were developed to predict high and low dengue incidence in order to provide timely forewarnings in the Philippines. METHODS: Model inputs were chosen based on studies indicating variables that may impact dengue incidence. The method first uses Fuzzy Association Rule Mining techniques to extract association rules from these historical epidemiological, environmental, and socio-economic data, as well as climate data indicating future weather patterns. Selection criteria were used to choose a subset of these rules for a classifier, thereby generating a Prediction Model. The models predicted high or low incidence of dengue in a Philippines province four weeks in advance. The threshold between high and low was determined relative to historical incidence data. PRINCIPAL FINDINGS: Model accuracy is described by Positive Predictive Value (PPV), Negative Predictive Value (NPV), Sensitivity, and Specificity computed on test data not previously used to develop the model. Selecting a model using the F0.5 measure, which gives PPV more importance than Sensitivity, gave these results: PPV = 0.780, NPV = 0.938, Sensitivity = 0.547, Specificity = 0.978. Using the F3 measure, which gives Sensitivity more importance than PPV, the selected model had PPV = 0.778, NPV = 0.948, Sensitivity = 0.627, Specificity = 0.974. The decision as to which model has greater utility depends on how the predictions will be used in a particular situation. CONCLUSIONS: This method builds prediction models for future dengue incidence in the Philippines and is capable of being modified for use in different situations; for diseases other than dengue; and for regions beyond the Philippines. The Philippines dengue prediction models predicted high or low incidence of dengue four weeks in advance of an outbreak with high accuracy, as measured by PPV, NPV, Sensitivity, and Specificity.

A data-driven epidemiological prediction method for dengue outbreaks using local and remote sensing data.

BACKGROUND: Dengue is the most common arboviral disease of humans, with more than one third of the world's population at risk. Accurate prediction of dengue outbreaks may lead to public health interventions that mitigate the effect of the disease. Predicting infectious disease outbreaks is a challenging task; truly predictive methods are still in their infancy. METHODS: We describe a novel prediction method utilizing Fuzzy Association Rule Mining to extract relationships between clinical, meteorological, climatic, and socio-political data from Peru. These relationships are in the form of rules. The best set of rules is automatically chosen and forms a classifier. That classifier is then used to predict future dengue incidence as either HIGH (outbreak) or LOW (no outbreak), where these values are defined as being above and below the mean previous dengue incidence plus two standard deviations, respectively. RESULTS: Our automated method built three different fuzzy association rule models. Using the first two weekly models, we predicted dengue incidence three and four weeks in advance, respectively. The third prediction encompassed a four-week period, specifically four to seven weeks from time of prediction. Using previously unused test data for the period 4-7 weeks from time of prediction yielded a positive predictive value of 0.686, a negative predictive value of 0.976, a sensitivity of 0.615, and a specificity of 0.982. CONCLUSIONS: We have developed a novel approach for dengue outbreak prediction. The method is general, could be extended for use in any geographical region, and has the potential to be extended to other environmentally influenced infections. The variables used in our method are widely available for most, if not all countries, enhancing the generalizability of our method.

Developing open source, self-contained disease surveillance software applications for use in resource-limited settings.

BACKGROUND: Emerging public health threats often originate in resource-limited countries. In recognition of this fact, the World Health Organization issued revised International Health Regulations in 2005, which call for significantly increased reporting and response capabilities for all signatory nations. Electronic biosurveillance systems can improve the timeliness of public health data collection, aid in the early detection of and response to disease outbreaks, and enhance situational awareness. METHODS: As components of its Suite for Automated Global bioSurveillance (SAGES) program, The Johns Hopkins University Applied Physics Laboratory developed two open-source, electronic biosurveillance systems for use in resource-limited settings. OpenESSENCE provides web-based data entry, analysis, and reporting. ESSENCE Desktop Edition provides similar capabilities for settings without internet access. Both systems may be configured to collect data using locally available cell phone technologies. RESULTS: ESSENCE Desktop Edition has been deployed for two years in the Republic of the Philippines. Local health clinics have rapidly adopted the new technology to provide daily reporting, thus eliminating the two-to-three week data lag of the previous paper-based system. CONCLUSIONS: OpenESSENCE and ESSENCE Desktop Edition are two open-source software products with the capability of significantly improving disease surveillance in a wide range of resource-limited settings. These products, and other emerging surveillance technologies, can assist resource-limited countries compliance with the revised International Health Regulations.

A meta-analysis of point-of-care laboratory tests in the diagnosis of novel 2009 swine-lineage pandemic influenza A (H1N1).

This paper reviews 14 published studies describing performance characteristics, including sensitivity and specificity, of commercially available rapid, point-of-care (POC) influenza tests in patients affected by an outbreak of a novel swine-related influenza A (H1N1) that was declared a pandemic in 2009. Although these POC tests were not intended to be specific for this pandemic influenza strain, the nonspecialized skills required and the timeliness of results make these POC tests potentially valuable for clinical and public health use. Pooled sensitivity and specificity for the POC tests studied were 68% and 81%, respectively, but published values were not homogeneous with sensitivities and specificities ranging from 10% to 88% and 51% to 100%, respectively. Pooled positive and negative likelihood ratios were 5.94 and 0.42, respectively. These results support current recommendations for use of rapid POC tests when H1N1 is suspected, recognizing that positive results are more reliable than negative results in determining infection, especially when disease prevalence is high.

Research priorities for syndromic surveillance systems response: consensus development using nominal group technique.

OBJECTIVE: To identify a set of fundable and practically feasible research priorities in the field of syndromic surveillance response on the basis of expert consensus. METHODS: The nominal group technique was used to structure an expert panel meeting in February 2009. Eleven national experts participated in the meeting, representing health departments at the city, county, state, and federal levels as well as academia and the military. RESULTS: The expert panel identified 3 research topics as consensus research priorities. These included the following: (1) How should different types of evidence and complementary data systems be integrated (merging data, visualizations)? (2) How can syndromic surveillance best be used in an electronic medical record environment? and (3) What criteria should be used to prioritize alerts? All identified research priorities were considered to be moderately highly fundable and feasible by an external group of experts with a record of obtaining grant funding in the field of biosurveillance. CONCLUSIONS: Prioritized research needs clustered around the common theme of how best to integrate diverse types and sources of information to inform action; thus, the major challenge that health departments are facing appears to be how to process abundant alert data from dissimilar sources. The nominal group technique in this study provided a method for systems' monitors to communicate their needs to the research community and can influence the commissioning of research by funding institutions.

A survey of usage protocols of syndromic surveillance systems by state public health departments in the United States.

OBJECTIVE: To broadly describe current syndromic surveillance systems in use throughout the United States and to provide basic descriptive information on responses to syndromic system signals. METHODS: Cross-sectional survey (telephone and e-mail) of state epidemiologists in all 50 states and the District of Columbia. RESULTS: Forty-one states participated in the survey for a response rate of 80 percent. Thirty-three states (80%) had at least one syndromic surveillance system in addition to BioSense operating within the state. Every state with an urban area at highest risk of a terrorist attack reported monitoring syndromic surveillance data, and a state's overall preparedness level was not related to the presence (or lack) of operational syndromic surveillance systems. The most common syndromic surveillance systems included BioSense (n = 20, 61%) and RODS (n = 13, 39%). Seventy-six percent of states with syndromic surveillance initiated investigations at the state level, 64 percent at the county level, and 45 percent at both the state and county levels. CONCLUSIONS: The majority of states reported using syndromic surveillance systems, with greatest penetration in those at highest risk for a terrorist attack. Most states used multiple systems and had varied methods (central and local) of responding to alerts, indicating the need for detailed response protocols.

Framework for the development of response protocols for public health syndromic surveillance systems: case studies of 8 US states.

OBJECTIVES: To describe current syndromic surveillance system response protocols in health departments from 8 diverse states in the United States and to develop a framework for health departments to use as a guide in initial design and/or enhancement of response protocols. METHODS: Case study design that incorporated in-depth interviews with health department staff, textual analysis of response plans, and a Delphi survey of syndromic surveillance response experts. RESULTS: All 8 states and 30 of the 33 eligible health departments agreed to participate (91% response rate). Fewer than half (48%) of surveyed health departments had a written response protocol, and health departments reported conducting in-depth investigations on fewer than 15% of syndromic surveillance alerts. A convened panel of experts identified 32 essential elements for inclusion in public health protocols for response to syndromic surveillance system alerts. CONCLUSIONS: Because of the lack of guidance, limited resources for development of response protocols, and few examples of syndromic surveillance detecting previously unknown events of public health significance, health departments have not prioritized the development and refinement of response protocols. Systems alone, however, are not effective without an organized public health response. The framework proposed here can guide health departments in creating protocols that will be standardized, tested, and relevant given their goals with such systems.

Enhancing Disease Surveillance Event Communication Among Jurisdictions.

Automated disease surveillance systems are becoming widely used by the public health community. However, communication among non-collocated and widely dispersed users still needs improvement. A web-based software tool for enhancing user communications was completely integrated into an existing automated disease surveillance system and was tested during two simulated exercises and operational use involving multiple jurisdictions. Evaluation of this tool was conducted by user meetings, anonymous surveys, and web logs. Public health officials found this tool to be useful, and the tool has been modified further to incorporate features suggested by user responses. Features of the automated disease surveillance system, such as alerts and time series plots, can be specifically referenced by user comments. The user may also indicate the alert response being considered by adding a color indicator to their comment. The web-based event communication tool described in this article provides a common ground for collaboration and communication among public health officials at different locations.

Bridging radiology and public health: the emerging field of radiologic public health informatics.

Radiology and public health have an emerging opportunity to collaborate, in which radiology's vast supply of imaging data can be integrated into public health information systems for epidemiologic assessments and responses to population health problems. Fueling the linkage of radiology and public health include (i) the transition from analog film to digital formats, enabling flexible use of radiologic data; (ii) radiology's role in imaging across nearly all medical and surgical subspecialties, which establishes a foundation for a consolidated and uniform database of images and reports for public health use; and (iii) the use of radiologic data to characterize disease patterns in a population occupying a geographic area at one time and to characterize disease progression over time via follow-up examinations. The backbone for this integration is through informatics projects such as Systematized Nomenclature of Medicine Clinical Terms and RadLex constructing terminology libraries and ontologies, as well as algorithms integrating data from the electronic health record and Digital Imaging and Communications in Medicine Structured Reporting. Radiology's role in public health is being tested in disease surveillance systems for outbreak detection and bioterrorism, such as the Electronic Surveillance System for the Early Notification of Community-based Epidemics. Challenges for radiologic public health informatics include refining the systems and user interfaces, adhering to privacy regulations, and strengthening collaborative relations among stakeholders, including radiologists and public health officials. Linking radiology with public health, radiologic public health informatics is a promising avenue through which radiology can contribute to public health decision making and health policy.

Pediatric patient asthma-related emergency department visits and admissions in Washington, DC, from 2001-2004, and associations with air quality, socio-economic status and age group.

BACKGROUND: The District of Columbia (DC) Department of Health, under a grant from the US Centers for Disease Control and Prevention, established an Environmental Public Health Tracking Program. As part of this program, the goals of this contextual pilot study are to quantify short-term associations between daily pediatric emergency department (ED) visits and admissions for asthma exacerbations with ozone and particulate concentrations, and broader associations with socio-economic status and age group. METHODS: Data included daily counts of de-identified asthma-related pediatric ED visits for DC residents and daily ozone and particulate concentrations during 2001-2004. Daily temperature, mold, and pollen measurements were also obtained. After a cubic spline was applied to control for long-term seasonal trends in the ED data, a Poisson regression analysis was applied to the time series of daily counts for selected age groups. RESULTS: Associations between pediatric asthma ED visits and outdoor ozone concentrations were significant and strongest for the 5-12 year-old age group, for which a 0.01-ppm increase in ozone concentration indicated a mean 3.2% increase in daily ED visits and a mean 8.3% increase in daily ED admissions. However, the 1-4 yr old age group had the highest rate of asthma-related ED visits. For 1-17 yr olds, the rates of both asthma-related ED visits and admissions increased logarithmically with the percentage of children living below the poverty threshold, slowing when this percentage exceeded 30%. CONCLUSION: Significant associations were found between ozone concentrations and asthma-related ED visits, especially for 5-12 year olds. The result that the most significant ozone associations were not seen in the age group (1-4 yrs) with the highest rate of asthma-related ED visits may be related to the clinical difficulty in accurately diagnosing asthma among this age group. We observed real increases in relative risk of asthma ED visits for children living in higher poverty zip codes versus other zip codes, as well as similar logarithmic relationships for visits and admissions, which implies ED over-utilization may not be a factor. These results could suggest designs for future epidemiological studies that include more information on individual exposures and other risk factors.