Measuring the accuracy of management effectiveness evaluations of protected areas.

Evaluating the effectiveness of protected area management to help improve management outcomes is becoming an increasingly common practice. The evaluation tools developed and implemented in over 100 countries generally rely on the expert judgements of protected area managers. Despite the growing collection and use of management effectiveness evaluation data, there have been no previous attempts to measure the accuracy of these data. We measured the accuracy of managers' judgements about the conditions in their reserves by collecting independent field data. We also assessed how accurately the evaluation tool reflected managers' views by conducting semi-structured interviews with 23 protected area managers from New South Wales, Australia. We found that managers made highly accurate judgements of the extent of a common weed species, Rubus fruticosus (blackberry), but often misinterpreted the scope, scale and timeframe of the evaluation. These framing effects can lead to error being introduced into the evaluation dataset, affecting the precision of evaluations such that they cannot be reliably compared among reserves. We suggest that the wording of evaluation questions needs to be explicit about the assessment frame to minimize the influence of framing effects on management effectiveness evaluations.

Managers consider multiple lines of evidence important for biodiversity management decisions.

Protected area managers often fail to use empirical evidence for their management decisions, yet it is unclear whether this arises from a lack of available data, difficulty in interpreting scientific information for management application, or because managers do not value science for their decisions. To better understand the use of evidence for management decisions, we asked protected area managers in Australia what information is important when making decisions, the types of evidence they find most valuable, and the types of evidence they have for their protected areas. Managers described a complex array of information needed for management decisions, with nine different factors representing decisions about individual management issues and how to prioritize management actions. While managers reported less access to empirical evidence than other sources, this is not because they do not value it, reporting it to be the most valuable source of evidence. Instead, they make up the shortfall in empirical evidence with experience and information synthesized from multiple lines of evidence, which can provide important context for their decisions. We conclude that managers value a diversity of evidence because they face complex conservation decisions. Therefore, while empirical evidence can play an important role, alone this cannot provide all the knowledge managers need.