Evaluation of animal and public health surveillance systems: a systematic review.

Disease surveillance programmes ought to be evaluated regularly to ensure they provide valuable information in an efficient manner. Evaluation of human and animal health surveillance programmes around the world is currently not standardized and therefore inconsistent. The aim of this systematic review was to review surveillance system attributes and the methods used for their assessment, together with the strengths and weaknesses of existing frameworks for evaluating surveillance in animal health, public health and allied disciplines. Information from 99 articles describing the evaluation of 101 surveillance systems was examined. A wide range of approaches for assessing 23 different system attributes was identified although most evaluations addressed only one or two attributes and comprehensive evaluations were uncommon. Surveillance objectives were often not stated in the articles reviewed and so the reasons for choosing certain attributes for assessment were not always apparent. This has the potential to introduce misleading results in surveillance evaluation. Due to the wide range of system attributes that may be assessed, methods should be explored which collapse these down into a small number of grouped characteristics by focusing on the relationships between attributes and their links to the objectives of the surveillance system and the evaluation. A generic and comprehensive evaluation framework could then be developed consisting of a limited number of common attributes together with several sets of secondary attributes which could be selected depending on the disease or range of diseases under surveillance and the purpose of the surveillance. Economic evaluation should be an integral part of the surveillance evaluation process. This would provide a significant benefit to decision-makers who often need to make choices based on limited or diminishing resources.

SERVAL: a new framework for the evaluation of animal health surveillance.

Animal health surveillance programmes may change in response to altering requirements or perceived weaknesses but are seldom subjected to any formal evaluation to ensure that they provide valuable information in an efficient manner. The literature on the evaluation of animal health surveillance systems is sparse, and those that are published may be unstructured and therefore incomplete. To address this gap, we have developed SERVAL, a SuRveillance EVALuation framework, which is novel and aims to be generic and therefore suitable for the evaluation of any animal health surveillance system. The inclusion of socio-economic criteria ensures that economic evaluation is an integral part of this framework. SERVAL was developed with input from a technical workshop of international experts followed by a consultation process involving providers and users of surveillance and evaluation data. It has been applied to a range of case studies encompassing different surveillance and evaluation objectives. Here, we describe the development, structure and application of the SERVAL framework. We discuss users' experiences in applying SERVAL to evaluate animal health surveillance systems in Great Britain.

Scrapie transmission in Britain: a recipe for a mathematical model.

Responses to an anonymous postal survey concerning scrapie are analysed. Risk factors associated with farms that have had scrapie are identified as size, geographical region, lambing practices and holding of certain breeds. Further analysis of farms that have scrapie only in bought-in animals reveals that such farms tend to breed a smaller proportion of their replacement animals than farms without scrapie. Farms that have had scrapie in home-bred animals have attributes associated with breeding many animals: large numbers of rams bought, few ewes bought, and many animals that are home-bred. The demography of British sheep farms as described by size, breeds, purchasing behaviour, age structure and proportion of animals that are home-bred is summarized. British farms with scrapie reveal certain special features: they have more sheep that are found dead, more elderly ewes and more cases of scab.

The flock-to-flock force of infection for scrapie in Britain.

A postal survey of British sheep farmers provided information on the proportion of farms that experienced their first case of scrapie in each year between 1962 and 1998. We found no evidence of a large increase in the proportion of scrapie-affected farms prior to, during or following the epidemic of BSE in British cattle. After correcting for between-farm heterogeneity in the probability of acquiring scrapie, we estimated the yearly between-flock force of infection since 1962. The current force of infection is estimated at approximately 0.0045 per farm per year and combined with a simple model of scrapie spread provides an estimate of the average duration of a scrapie outbreak on an individual farm. Considering all farms, the average outbreak lasts for five years, but if only those farms that have cases in animals born on the farm are considered, it lasts 15 years. We use these parameter estimates to compare the proportion of farms with scrapie in time periods of different lengths. In the survey, 2.7% of farms had a case in 1998. The 5.3% of farms reporting having a case between 1993 and 1997 is consistent with the hypothesis that the scrapie force of infection remained constant over this period.

Assessing the efficacy of a ram-genotyping programme to reduce susceptibility to scrapie in Great Britain.

Susceptibility to clinical scrapie is associated with polymorphisms in the PrP gene. The 'ARR' allele of this gene reduces susceptibility to clinical disease caused by all known strains of the transmissible spongiform encephalopathy (TSE) agent. The British government proposes to use a ram-genotyping scheme to breed genetic resistance to clinical scrapie into the national sheep population. We considered how best to target limited genotyping resources to achieve the maximum rate of genotype evolution. We created a metapopulation model of the British sheep industry, which includes the major pure-breeds of sheep and the cross-breeds produced by crossing these pure-bred animals. The main criterion for assessing the efficacy of different strategies was the time taken to increase the prevalence of the ARR allele in the slaughter-lamb population. Our model predicted that the most-effective strategy would be to target genotyping to those rams used for pure-breeding (i.e. mated with the same breed of ewe). This strategy was compared to two further strategies, in which the proportion of rams genotyped in each breed depended on the prevalence of the ARR/ARR genotype in that breed. A policy in which the proportion of animals genotyped is reduced as the ARR prevalence in that breed increases is efficient. The most-effective policy was targeting the hill sector in the early years and gradually switching to genotyping more terminal-sire and longwool rams as the resistance of the hill sector increases.

A stochastic model to estimate the prevalence of scrapie in Great Britain using the results of an abattoir-based survey.

In 1997/1998, an abattoir survey was conducted to determine the likely exposure of the human population to transmissible spongiform encephalopathy (TSE) infection in sheep submitted for slaughter in Great Britain. The survey examined brain material from 2809 sheep processed through British abattoirs. Sampling was targeted by age: 45% of animals tested were > or =15 months old. All samples of adequate quality (98%) were tested for signs of scrapie infection using histopathology and scrapie-associated fibril (SAF) detection and 500 were tested using immunohistochemistry (IHC). No conclusive positive animals were found using either histology or IHC. Ten animals were positive by SAF. Standard statistical analyses suggest (with 95% confidence) that the prevalence of detectable (by histopathology) infection in the slaughter population was < or =0.11%. However, the incubation period of scrapie is long (usually around 2-3 years) and none of the tests used in the survey is capable of detecting scrapie infection in the early stages of infection. We present an age-structured stochastic model incorporating parameters for the incubation period of scrapie, prevalence of infection by age and test sensitivity. Using the model, we demonstrate that the negative results obtained for all samples using IHC and histopathology are consistent with a true prevalence of infection in the slaughter population of up to 11%. This suggests that up to 300 of the animals tested might have been infected but the infection was not sufficiently advanced in these animals to be detectable by IHC or histopathology. The survey was designed to detect a prevalence of 1% with a precision of +/-0.5% and a confidence level of 95% in each age group assuming that diagnostic tests were 100% specific and sensitive from a known stage in the incubation period. The results of the model demonstrate that to estimate a true prevalence of scrapie infection of 1% with an accuracy of +/-0.5% would have required a far larger sample size. An accurate estimate of the required sample size is complicated by uncertainty about test sensitivity and the underlying infection dynamics of scrapie. A pre-requisite for any future abattoir survey is validation of the diagnostic tests used in relation to both stage of incubation and genotype. Sampling in the <15-month age group was of no value in this survey because the diagnostic tests used were thought to be ineffective in most of the animals in this age group.

A review of the epidemiology of scrapie in sheep.

The aim of this review is to summarise and evaluate the data available about the aetiology of scrapie in naturally affected sheep flocks, particularly data concerning the possible transmission of infection between related animals. The author examines data taken from various relevant studies carried out over the last thirty years. The main conclusions are that scrapie is an infectious disease with a genetic influence on the incubation period. The increased risk of disease in the offspring of affected animals is thought to be largely the result of increased genetic susceptibility, with a large proportion of the cases occurring in high-incidence flocks being the result of horizontal transmission of infection.

Descriptive epidemiology of scrapie in Great Britain: results of a postal survey.

In 1998, a questionnaire was sent to 11,554 British sheep farmers to determine how many believed that scrapie cases had occurred in their flock; 61.4 per cent of them responded anonymously. The results indicated that 14.9 per cent of farmers with more than 30 breeding ewes thought that they had ever experienced scrapie in their flock and 2.7 per cent thought that they had had cases in the past 12 months. A comparison of these results with the number of farmers reporting suspect scrapie cases to MAFF, in accordance with the statutory requirement, suggests that only 13 per cent of farmers who suspect that they may have cases of scrapie are currently reporting them. Scrapie occurred in all regions of the country but there was an apparent regional variation. Larger farms and those with purebred sheep appeared to be at greater risk of having cases. Other differences between affected and unaffected farms included lambing practices and sheep purchasing policy. On the majority of farms the first case occurred in a purchased animal. The survey also revealed a need for the provision of further information about scrapie to farmers.

An investigation of risk factors for cases of bovine spongiform encephalopathy born after the introduction of the 'feed ban'.

Bovine spongiform encephalopathy (BSE) occurred in cattle in Great Britain after the inclusion of protein derived from infected tissues in their feed, and the incidence of the disease has been reduced by the introduction of legislation to prevent the inclusion of such protein in ruminant feed. This paper describes a case-control study designed to investigate whether there is any evidence for direct transmission of infection to cattle born after the introduction of this legislation. The offspring of animals that were subsequently affected with BSE were not found significantly more often among the cases. There was a statistically significant risk for animals born up to three days after a subsequently affected animal calved, but it may not indicate a causal association. Even after adjusting for an animal's exposure to infected animals that calved but would have been culled from the herd before developing clinical signs of BSE these routes of transmission could not account for the majority of cases born after the introduction of the legislation. A between herd comparison is suggested as a method of investigating alternative sources of infection.

Bovine spongiform encephalopathy: aspects of the clinical picture and analyses of possible changes 1986-1990.

A standard questionnaire was used to record the presence of specific clinical signs reported for histopathologically confirmed cases of bovine spongiform encephalopathy observed before June 30, 1990, and the frequencies of these signs were analysed. The signs most frequently recorded were apprehension, hyperaesthesia and ataxia, and there were variations in the frequency with which some signs were recorded in animals observed at different times during the epidemic. These variations were considered to be the result of differences between observers and differences in the duration of the illness, rather than a change in the clinical picture, which could possibly have occurred with a change in the nature of the agent to which the cattle had been exposed, or to a change in their response to the agent.

Bovine spongiform encephalopathy: epidemiological features 1985 to 1990.

Following the identification of bovine spongiform encephalopathy (BSE) in the British cattle population in 1986 epidemiological studies were launched. This paper provides an updated account of the epidemiological features of BSE from 1985, when the first cases, based on clinical histories, occurred, until 1990. The number of cases up to December 1989 represents an annual incidence of 3.9 confirmed cases per 1000 adult animals in Great Britain. Many more dairy herds were affected than beef suckler herds, a difference attributable to the difference in feeding practices between the two herd types. The geographical variation in incidence previously described has persisted with the highest incidence in the south and east of England. Other features of the epidemiology, including the low within-herd incidence, remained unaltered from the earlier findings. The results support the previously suggested hypothesis that the outbreak of BSE was due to the sudden exposure of the cattle population to a scrapie-like agent in 1981/82. There was no evidence of direct transmission between cattle during the period considered.

Scrapie surveillance in Great Britain: results of an abattoir survey, 1997/98.

A randomised sample of 2,809 apparently healthy sheep, 55 per cent of them less than 15 months of age, which were slaughtered for human consumption at abattoirs in Great Britain in 1997/98, was taken to establish the prevalence of scrapie infection. The medulla oblongata of each sheep was examined histopathologically at the level of the obex, and fresh brain tissue was examined for scrapie-associated fibrils (SAF) to establish whether there was evidence of scrapie. In addition, histological sections of the medulla from 500 of the sheep were immunostained with an antiserum to PrP, and the same technique was also applied to any animal found positive or inconclusive by the histological or SAF examinations. Any sheep which was positive by any of these diagnostic methods was also examined by Western immunoblotting, for the detection of the disease-specific protein PrP(Sc). A total of 2,798 sheep (99.6 per cent) were negative by all the methods applied. Ten animals were SAF-positive but negative by all the other methods, and in one animal there was immunohistochemical staining which could not be interpreted unequivocally as disease-specific. A mathematical model was used to estimate the prevalence of scrapie infection in the national slaughtered sheep population which would be consistent with these results. By this model, the absence of unequivocally substantiated cases of scrapie in the sample was consistent with a prevalence of infection in the slaughter population of up to 11 per cent.

Proposed terms and concepts for describing and evaluating animal-health surveillance systems.

The information provided by animal-health surveillance helps to reduce the impact of animal diseases. The widespread movement of animals and their products around the world results in an increasing risk that disease will spread. There is, therefore, a need for exchange between countries of comparable information about disease incidence; the exchange must be based on a common understanding of surveillance approaches and how surveillance systems are designed and implemented. Establishing agreed-upon definitions of surveillance terms would be a first step in achieving this standardisation, and will enhance transparency and confidence. To this end, a workshop was held with the aim of agreeing upon key terms and concepts for animal-health surveillance. In this paper, we describe the methods used at the workshop and summarise the discussions. A complete list of all the proposed definitions including lists of characteristics that can be used to describe surveillance activities and attributes for evaluation of surveillance is available in the workshop report (available at http://www.defra.gov.uk/ahvla-en/disease-control/surveillance/icahs-workshop/). Some important issues were highlighted during these discussions; of particular note was the importance of economic efficiency as an evaluation attribute. Some remaining inconsistencies in the proposed use of terms are highlighted (including the definition of 'risk-based surveillance' and the use of the term 'event-based surveillance').

Evaluation of methods for measuring coverage and representativeness of an early-warning disease surveillance system.

Early-warning surveillance provides an essential component of the evidence required to protect animal health. Assessing the proportion of the population included in surveillance systems (coverage) provides a measure of the effectiveness of early-warning surveillance, and contributes to ensuring that these systems are efficient. This paper describes an investigation of methods used for assessing the coverage and representativeness of the 'FarmFile' early-warning surveillance system. This system uses information collected with samples submitted to diagnostic laboratories by private veterinary practitioners in England and Wales. Available data on pig holdings and veterinary practices in four English counties, selected to represent a range of diverse population characteristics, were supplemented using surveys of veterinary practices. Coverage assessments were based on submissions made to FarmFile in 2009. The proportion of holdings covered varied from 5-62 per cent in Devon and Cumbria, and 16-97 per cent in Norfolk and East Riding of Yorkshire. The results suggest that while the proportion of individual pigs covered by the current early-warning surveillance system is high, small and breeding-only holdings in some regions may be poorly covered. Coverage assessments vary depending on the methods used for their assessment, and multiple assessment methods can provide a 'range' within which coverage lies.

Estimation of Salmonella prevalence in UK egg-laying holdings.

As part of an EU-wide programme to reduce the prevalence of Salmonella in commercial egg-laying holdings, the EU has set for the UK an annual target of 10% reduction in the prevalence of Salmonella Enteritidis and Salmonella Typhimurium in commercial egg-laying holdings. To assist in demonstrating such a reduction, it is very important to obtain an accurate as possible baseline prevalence for Salmonella. The objective of this study was to provide a baseline estimate of the Salmonella prevalence in egg-laying holdings in the UK. Data from an EU baseline survey for Salmonella in UK commercial egg-laying flocks were therefore analysed using Bayesian methods, taking into account the sampling of only 1 flock per holding and estimates of the test sensitivity of the methods used in the EU baseline survey. In addition, in the UK the majority of the eggs come from farms which have participated in voluntary monitoring programmes for Salmonella since the early 1990s, and this data was also used, along with a prior estimate of the test sensitivity of voluntary surveillance. Results indicated that a true prevalence 14% for Salmonella Enteriditis and Typhimurium, and 18% for all serovars, both of these estimates being higher than has previously been reported from the EU baseline survey data. It is also shown that the sensitivity of voluntary surveillance is low, and it will therefore be important to compare results from "official" and "non-official" samples to check that the sampling performed in the National Control Plan is as sensitive as expected.

Transmission dynamics and epidemiology of BSE in British cattle.

A comprehensive analysis of the bovine spongiform encephalopathy (BSE) epidemic in cattle in Great Britain assesses past, present and future patterns in the incidence of infection and disease, and allows a critical appraisal of different culling policies for eradication of the disease.