Analytical challenges for emerging public health surveillance.

The root of effective disease control and prevention is an informed understanding of the epidemiology of a particular disease based on sound scientific interpretation of evidence. Such evidence must frequently be transformed from raw data into consumable information before it can be used for making decisions, determining policy, and conducting programs. However, the work of building such evidence in public health practice--doing the right thing at the right time--is essentially hidden from view. Surveillance involves acquiring, analyzing, and interpreting data and information from several sources across various systems. Achieving the goals and objectives of surveillance investments requires attention to analytic requirements of such systems. The process requires computer programming, statistical reasoning, subject matter expertise, often modeling, and effective communication skills.

Issues in applied statistics for public health bioterrorism surveillance using multiple data streams: research needs.

The objective of this report is to provide a basis to inform decisions about priorities for developing statistical research initiatives in the field of public health surveillance for emerging threats. Rapid information system advances have created a vast opportunity of secondary data sources for information to enhance the situational and health status awareness of populations. While the field of medical informatics and initiatives to standardize healthcare-seeking encounter records continue accelerating, it is necessary to adapt analytic and statistical methodologies to mature in sync with sibling information science technologies. One major right-of-passage for statistical inference is to advance the optimal application of analytic methodologies for using multiple data streams in detecting and characterizing public health population events of importance. This report first describes the problem in general and the data context, then delineates more specifically the practical nature of the problem and the related issues. Approaches currently applied to data with time-series, statistical process control and traditional inference concepts are described with examples in the section on Statistics and the Role of the Analytic Surveillance Data Monitor. These are the techniques that are providing substance to surveillance professionals and enabling use of multiple data streams. The next section describes use of a more complex approach that takes temporal as well as spatial dimensions into consideration for detection and situational awareness regarding event distributions. The space-time statistic has successfully been used to detect and track public health events of interest. Important research questions which are summarized at the end of this report are described in more detail with respect to the methodological application in the respective sections. This was thought to help elucidate the research requirements as summarized later in the report. Following the description of the space-time scan statistical application; this report extends to a less traditional area of promise given what has been observed in recent application of analytic methods. Bayesian networks (BNs) represent a conceptual step with advantages of flexibility for the public health surveillance community. Progression from traditional to the more extending statistical concepts in the context of the dynamic status quo of responsibility and challenge, leads to a conclusion consisting of categorical research needs. The report is structured by design to inform judgment about how to build on practical systems to achieve better analytic outcomes for public health surveillance. There are references to research issues throughout the sections with a summarization at the end, which also includes items previously unmentioned in the report.

Using simulation to assess the sensitivity and specificity of a signal detection tool for multidimensional public health surveillance data.

The objective of the work described in this paper is to develop a means for characterizing the validity of an empirical methodology for detecting signals potentially related to complicated adverse event (AE) coding terms in multidimensional public health surveillance data. The signal detection tool under evaluation is the multi-item gamma Poisson shrinkage (MGPS) estimation program. We were interested in its potential application to passive surveillance system monitoring, to screen for 'signals' of complicated adverse event coding terms (AE terms) in complex and noisy data. The research was to design and produce a flexible and user-friendly utility for probabilistically defining complicated signals in a database, iterating large numbers of applications of the MGPS detection algorithm and establishing proportions of correct detection events. We sought to establish the specificity of the MGPS by developing a random background using a gradient that ranged from rigorous (but not very relevant) to relevant (but noisy). To establish the sensitivity, signals were defined based on recognized public health issues of interest (such as the introduction of a new vaccine into the population). Methods of representing a signal included a simple pair-wise association consisting of a new vaccine and one AE term, as well as a more realistic complex of multiple AE terms comprising a 'syndrome'. A web application has been developed to create and insert signals with user-defined probabilities in multiple iterations of simulated random background data. Three forms of simulated data based on the vaccine adverse event reporting system (VAERS) cumulative spontaneous database were defined to serve as background noise against which to contrast introduced vaccine adverse event signals: (1) completely random associations between vaccines and AE terms, (2) random associations of vaccine sets and AE term sets preserving naturally observed vaccine co-occurrences and AE term co-occurrences and (3) samples from the actual VAERS data as reported. Rates of detection by the MGPS algorithm can be established for specific signal patterns at varying probabilistic intensities in a choice of random background data forms. Knowing these rates is important for determining the degree of response to an MGPS signal detection event in 'live' data.

Random error and undercounting in birth defects surveillance data: implications for inference.

BACKGROUND: There has been an ongoing debate among birth defects investigators about whether or not to publish estimates of rates of birth defects with confidence intervals to allow for comparisons of rates across regions and time. A major impediment in resolving this debate has been the lack of a framework for quantifying uncertainties in the data that can be applied uniformly to birth defects surveillance programs. This report presents an overview of random error and ascertainment bias in birth defects surveillance data, and of the implications of these errors for estimation and comparisons of birth defects rates. METHODS: We consider when confidence intervals can be used as part of a strategy to make inference on rates, as well as ratios of or differences between two rates. Worth noting is that confidence intervals only address random error in the data. In the presence of undercounting of cases, estimation of rates and confidence intervals requires knowledge or an estimate of the extent of underascertainment. Rate estimates and confidence intervals that ignore such bias can be misleading. However, if it is reasonable to assume that the ascertainment bias is constant over time (or across regions), then it is possible to make valid comparisons of rates over time (or across regions) using ratio or difference estimators, even when lack of knowledge of the extent of undercounting makes estimating the absolute rate and its confidence interval problematic. Finally, sensitivity analyses can use confidence limits to determine the difference in ascertainment bias necessary to explain an apparent difference in rates. CONCLUSION: Because birth defects surveillance systems have evolved in the absence of agreed upon standards to guide the process, it is difficult to determine the extent to which the variability in rates of birth defects across programs or over time is real or due to differences in surveillance methods. Efforts to develop standards for birth defects surveillance may help to minimize the variability in prevalence of birth defects due to differences in case ascertainment methods and allow for evaluations of real temporal and spatial variations in environmental effects. In the meantime, if comparisons of rates need to be made to address public health concerns, it would be prudent to conduct only such comparisons between regions or across time when the degree of case ascertainment can be assumed to be relatively constant across regions and time.