Policy options for endgame planning in tobacco control: a simulation modelling study.

OBJECTIVE: To investigate the potential impacts of several tobacco control interventions on adult daily smoking prevalence in the Australian state of Queensland, using a system dynamics model codeveloped with local and national stakeholders. METHODS: Eight intervention scenarios were simulated and compared with a reference scenario (business as usual), in which all tobacco control measures currently in place are maintained unchanged until the end of the simulation period (31 December 2037). FINDINGS: Under the business as usual scenario, adult daily smoking prevalence is projected to decline from 11.8% in 2017 to 5.58% in 2037. A sustained 50% increase in antismoking advertising exposure from 2018 reduces projected prevalence in 2037 by 0.80 percentage points. Similar reductions are projected with the introduction of tobacco wholesaler and retailer licensing schemes that either permit or prohibit tobacco sales by alcohol-licensed venues (0.65 and 1.73 percentage points, respectively). Increasing the minimum age of legal supply of tobacco products substantially reduces adolescent initiation, but has minimal impact on smoking prevalence in the adult population over the simulation period. Sustained reductions in antismoking advertising exposure of 50% and 100% from 2018 increase projected adult daily smoking prevalence in 2037 by 0.88 and 1.98 percentage points, respectively. CONCLUSIONS: These results suggest that any prudent approach to endgame planning should seek to build on rather than replace existing tobacco control measures that have proved effective to date. Additional interventions that can promote cessation are expected to be more successful in reducing smoking prevalence than interventions focussing exclusively on preventing initiation.

Systems modelling and simulation to inform strategic decision making for suicide prevention in rural New South Wales (Australia).

BACKGROUND: The need to understand and respond to the unique characteristics and drivers of suicidal behaviour in rural areas has been enabled through the Australian Government's 2015 mental health reforms facilitating a move to an evidence-based, regional approach to suicide prevention. However, a key challenge has been the complex decision-making environment and lack of appropriate tools to facilitate the use of evidence, data and expert knowledge in a way that can inform contextually appropriate strategies that will deliver the greatest impact. This paper reports the co-development of an advanced decision support tool that enables regional decision makers to explore the likely impacts of their decisions before implementing them in the real world. METHODS: A system dynamics model for the rural and remote population catchment of Western New South Wales was developed. The model was based on defined pathways to mental health care and suicidal behaviour and reproduced historic trends in the incidence of attempted suicide (self-harm hospitalisations) and suicide deaths in the region. A series of intervention scenarios were investigated to forecast their impact on suicidal behaviour over a 10-year period. RESULTS: Post-suicide attempt assertive aftercare was forecast to deliver the greatest impact, reducing the numbers of self-harm hospitalisations and suicide deaths by 5.65% (95% interval, 4.87-6.42%) and 5.45% (4.68-6.22%), respectively. Reductions were also projected for community support programs (self-harm hospitalisations: 2.83%, 95% interval 2.23-3.46%; suicide deaths: 4.38%, 95% interval 3.78-5.00%). Some scenarios produced unintuitive impacts or effect sizes that were significantly lower than what has been anticipated under the traditional evidence-based approach to suicide prevention and provide an opportunity for learning. CONCLUSION: Systems modelling and simulation offers significant potential for regional decision makers to better understand and respond to the unique characteristics and drivers of suicidal behaviour in their catchments and more effectively allocate limited health resources.

Bringing new tools, a regional focus, resource-sensitivity, local engagement and necessary discipline to mental health policy and planning.

BACKGROUND: While reducing the burden of mental and substance use disorders is a global challenge, it is played out locally. Mental disorders have early ages of onset, syndromal complexity and high individual variability in course and response to treatment. As most locally-delivered health systems do not account for this complexity in their design, implementation, scale or evaluation they often result in disappointing impacts. DISCUSSION: In this viewpoint, we contend that the absence of an appropriate predictive planning framework is one critical reason that countries fail to make substantial progress in mental health outcomes. Addressing this missing infrastructure is vital to guide and coordinate national and regional (local) investments, to ensure limited mental health resources are put to best use, and to strengthen health systems to achieve the mental health targets of the 2015 Sustainable Development Goals. Most broad national policies over-emphasize provision of single elements of care (e.g. medicines, individual psychological therapies) and assess their population-level impact through static, linear and program logic-based evaluation. More sophisticated decision analytic approaches that can account for complexity have long been successfully used in non-health sectors and are now emerging in mental health research and practice. We argue that utilization of advanced decision support tools such as systems modelling and simulation, is now required to bring a necessary discipline to new national and local investments in transforming mental health systems. CONCLUSION: Systems modelling and simulation delivers an interactive decision analytic tool to test mental health reform and service planning scenarios in a safe environment before implementing them in the real world. The approach drives better decision-making and can inform the scale up of effective and contextually relevant strategies to reduce the burden of mental disorder and enhance the mental wealth of nations.

'Turning the tide' on hyperglycemia in pregnancy: insights from multiscale dynamic simulation modeling.

INTRODUCTION: Hyperglycemia in pregnancy (HIP, including gestational diabetes and pre-existing type 1 and type 2 diabetes) is increasing, with associated risks to the health of women and their babies. Strategies to manage and prevent this condition are contested. Dynamic simulation models (DSM) can test policy and program scenarios before implementation in the real world. This paper reports the development and use of an advanced DSM exploring the impact of maternal weight status interventions on incidence of HIP. METHODS: A consortium of experts collaboratively developed a hybrid DSM of HIP, comprising system dynamics, agent-based and discrete event model components. The structure and parameterization drew on a range of evidence and data sources. Scenarios comparing population-level and targeted prevention interventions were simulated from 2018 to identify the intervention combination that would deliver the greatest impact. RESULTS: Population interventions promoting weight loss in early adulthood were found to be effective, reducing the population incidence of HIP by 17.3% by 2030 (baseline ('business as usual' scenario)=16.1%, 95% CI 15.8 to 16.4; population intervention=13.3%, 95% CI 13.0 to 13.6), more than targeted prepregnancy (5.2% reduction; incidence=15.3%, 95% CI 15.0 to 15.6) and interpregnancy (4.2% reduction; incidence=15.5%, 95% CI 15.2 to 15.8) interventions. Combining targeted interventions for high-risk groups with population interventions promoting healthy weight was most effective in reducing HIP incidence (28.8% reduction by 2030; incidence=11.5, 95% CI 11.2 to 11.8). Scenarios exploring the effect of childhood weight status on entry to adulthood demonstrated significant impact in the selected outcome measure for glycemic regulation, insulin sensitivity in the short term and HIP in the long term. DISCUSSION: Population-level weight reduction interventions will be necessary to 'turn the tide' on HIP. Weight reduction interventions targeting high-risk individuals, while beneficial for those individuals, did not significantly impact forecasted HIP incidence rates. The importance of maintaining interventions promoting healthy weight in childhood was demonstrated.

The Impact of Reducing Psychiatric Beds on Suicide Rates.

There has been ongoing debate regarding the impact of reductions in psychiatric beds on suicide rates, and the potential effect of reallocation of acute hospital funding to community-based mental health programs and services. Computer simulation offers significant value in advancing such debate by providing a robust platform for exploring strategic resource allocation scenarios before they are implemented in the real world. We report an application that demonstrates a threshold effect of cuts to psychiatric beds on suicide rates and the role of context specific variations in population, behavioral, and service use dynamics in determining where that threshold lies. Findings have important implications for regional decision-making regarding resource allocation for suicide prevention.

Turning conceptual systems maps into dynamic simulation models: An Australian case study for diabetes in pregnancy.

BACKGROUND: System science approaches are increasingly used to explore complex public health problems. Quantitative methods, such as participatory dynamic simulation modelling, can mobilise knowledge to inform health policy decisions. However, the analytic and practical steps required to turn collaboratively developed, qualitative system maps into rigorous and policy-relevant quantified dynamic simulation models are not well described. This paper reports on the processes, interactions and decisions that occurred at the interface between modellers and end-user participants in an applied health sector case study focusing on diabetes in pregnancy. METHODS: An analysis was conducted using qualitative data from a participatory dynamic simulation modelling case study in an Australian health policy setting. Recordings of participatory model development workshops and subsequent meetings were analysed and triangulated with field notes and other written records of discussions and decisions. Case study vignettes were collated to illustrate the deliberations and decisions made throughout the model development process. RESULTS: The key analytic objectives and decision-making processes included: defining the model scope; analysing and refining the model structure to maximise local relevance and utility; reviewing and incorporating evidence to inform model parameters and assumptions; focusing the model on priority policy questions; communicating results and applying the models to policy processes. These stages did not occur sequentially; the model development was cyclical and iterative with decisions being re-visited and refined throughout the process. Storytelling was an effective strategy to both communicate and resolve concerns about the model logic and structure, and to communicate the outputs of the model to a broader audience. CONCLUSION: The in-depth analysis reported here examined the application of participatory modelling methods to move beyond qualitative conceptual mapping to the development of a rigorously quantified and policy relevant, complex dynamic simulation model. The analytic objectives and decision-making themes identified provide guidance for interpreting, understanding and reporting future participatory modelling projects and methods.

Resolving the paradox of increased mental health expenditure and stable prevalence.

A doubling of Australian expenditure on mental health services over two decades, inflation-adjusted, has reduced prevalence of neither psychological distress nor mental disorders. Low rates of help-seeking, and inadequate and inequitable delivery of effective care may explain this partially, but not fully. Focusing on depressive disorders, drawing initially on ideas from the work of philosopher and socio-cultural critic Ivan Illich, we use evidence-based medicine statistics and simulation modelling approaches to develop testable hypotheses as to how iatrogenic influences on the course of depression may help explain this seeming paradox. Combined psychological treatment and antidepressant medication may be available, and beneficial, for depressed people in socioeconomically advantaged areas. But more Australians with depression live in disadvantaged areas where antidepressant medication provision without formal psychotherapy is more typical; there also are urban/non-urban disparities. Depressed people often engage in self-help strategies consistent with psychological treatments, probably often with some benefit to these people. We propose then, if people are encouraged to rely heavily on antidepressant medication only, and if they consequently reduce spontaneous self-help activity, that the benefits of the antidepressant medication may be more than offset by reductions in beneficial effects as a consequence of reduced self-help activity. While in advantaged areas, more comprehensive service delivery may result in observed prevalence lower than it would be without services, in less well-serviced areas, observed prevalence may be higher than it would otherwise be. Overall, then, we see no change. If the hypotheses receive support from the proposed research, then implications for service prioritisation and delivery could include a case for wider application of recovery-oriented practice. Critically, it would strengthen the case for action to correct inequities in the delivery of psychological treatments for depression in Australia so that combined psychological therapy and antidepressant medication, accessible and administered within an empowering framework, should be a nationally implemented standard.

The impact of strengthening mental health services to prevent suicidal behaviour.

OBJECTIVE: Successive suicide prevention frameworks and action plans in Australia and internationally have called for improvements to mental health services and enhancement of workforce capacity. However, there is debate regarding the priorities for resource allocation and the optimal combination of mental health services to best prevent suicidal behaviour. This study investigates the potential impacts of service capacity improvements on the incidence of suicidal behaviour in the Australian context. METHODS: A system dynamics model was developed to investigate the optimal combination of (1) secondary (acute) mental health service capacity, (2) non-secondary (non-acute) mental health service capacity and (3) resources to re-engage those lost to services on the incidence of suicidal behaviour over the period 2018-2028 for the Greater Western Sydney (Australia) population catchment. The model captured population and behavioural dynamics and mental health service referral pathways and was validated using population survey and administrative data, evidence syntheses and an expert stakeholder group. RESULTS: Findings suggest that 28% of attempted suicide and 29% of suicides could be averted over the forecast period based on a combination of increases in (1) hospital staffing (with training in trauma-informed care), (2) non-secondary health service capacity, (3) expansion of mental health assessment capacity and (4) re-engagement of at least 45% of individuals lost to services. Reduction in the number of available psychiatric beds by 15% had no substantial impact on the incidence of attempted suicide and suicide over the forecast period. CONCLUSION: This study suggests that more than one-quarter of suicides and attempted suicides in the Greater Western Sydney population catchment could potentially be averted with a combination of increases to hospital staffing and non-secondary (non-acute) mental health care. Reductions in tertiary care services (e.g. psychiatric hospital beds) in combination with these increases would not adversely affect subsequent incidence of suicidal behaviour.

Decision makers' experience of participatory dynamic simulation modelling: methods for public health policy.

BACKGROUND: Systems science methods such as dynamic simulation modelling are well suited to address questions about public health policy as they consider the complexity, context and dynamic nature of system-wide behaviours. Advances in technology have led to increased accessibility and interest in systems methods to address complex health policy issues. However, the involvement of policy decision makers in health-related simulation model development has been lacking. Where end-users have been included, there has been limited examination of their experience of the participatory modelling process and their views about the utility of the findings. This paper reports the experience of end-user decision makers, including senior public health policy makers and health service providers, who participated in three participatory simulation modelling for health policy case studies (alcohol related harm, childhood obesity prevention, diabetes in pregnancy), and their perceptions of the value and efficacy of this method in an applied health sector context. METHODS: Semi-structured interviews were conducted with end-user participants from three participatory simulation modelling case studies in Australian real-world policy settings. Interviewees were employees of government agencies with jurisdiction over policy and program decisions and were purposively selected to include perspectives at different stages of model development. RESULTS: The 'co-production' aspect of the participatory approach was highly valued. It was reported as an essential component of building understanding of the modelling process, and thus trust in the model and its outputs as a decision-support tool. The unique benefits of simulation modelling included its capacity to explore interactions of risk factors and combined interventions, and the impact of scaling up interventions. Participants also valued simulating new interventions prior to implementation in the real world, and the comprehensive mapping of evidence and its gaps to prioritise future research. The participatory aspect of simulation modelling was time and resource intensive and therefore most suited to high priority complex topics with contested options for intervening. CONCLUSION: These findings highlight the value of a participatory approach to dynamic simulation modelling to support its utility in applied health policy settings.

A decision support tool to inform local suicide prevention activity in Greater Western Sydney (Australia).

OBJECTIVES: This study describes the development of a decision support tool to identify the combination of suicide prevention activities and service priorities likely to deliver the greatest reductions in suicidal behaviour in Western Sydney (Australia) over the period 2018-2028. METHODS: A dynamic simulation model for the WentWest - Western Sydney Primary Health Network population-catchment was developed in partnership with primary health network stakeholders based on defined pathways to mental health care and suicidal behaviour, and which represented the current incidence of suicide and attempted suicide in Western Sydney. A series of scenarios relating to potential suicide prevention activities and service priorities identified by primary health network stakeholders were investigated to identify the combination of interventions associated with the largest reductions in the forecast number of attempted suicide and suicide cases for a 10-year follow-up period. RESULTS: The largest number of cases averted for both suicide and attempted suicide was associated with (1) post-suicide attempt assertive aftercare (6.1% for both attempted suicide and suicide), (2) improved community support and reductions in psychological distress in the community (5.1% for attempted suicide and 14.8% for suicide), and (3) reductions in the proportion of those lost to services following a mental health service contact (10.5% for both attempted suicide and suicide). In combination, these interventions were forecast to avert approximately 29.7% of attempted suicides and 37.1% of suicides in the primary health network catchment over the 10-year period. CONCLUSION: This study demonstrates the utility of dynamic simulation models, co-designed with multi-disciplinary stakeholder groups, to capture and analyse complex mental health and suicide prevention regional planning problems. The model can be used by WentWest - Western Sydney Primary Health Network as a decision support tool to guide the commissioning of future service activity, and more efficiently frame the monitoring and evaluation of interventions as they are implemented in Western Sydney.

Impacts of licensed premises trading hour policies on alcohol-related harms.

BACKGROUND AND AIM: Evaluations of alcohol policy changes demonstrate that restriction of trading hours of both 'on'- and 'off'-licence venues can be an effective means of reducing rates of alcohol-related harm. Despite this, the effects of different trading hour policy options over time, accounting for different contexts and demographic characteristics, and the common co-occurrence of other harm reduction strategies in trading hour policy initiatives, are difficult to estimate. The aim of this study was to use dynamic simulation modelling to compare estimated impacts over time of a range of trading hour policy options on various indicators of acute alcohol-related harm. METHODS: An agent-based model of alcohol consumption in New South Wales, Australia was developed using existing research evidence, analysis of available data and a structured approach to incorporating expert opinion. Five policy scenarios were simulated, including restrictions to trading hours of on-licence venues and extensions to trading hours of bottle shops. The impact of the scenarios on four measures of alcohol-related harm were considered: total acute harms, alcohol-related violence, emergency department (ED) presentations and hospitalizations. RESULTS: Simulation of a 3 a.m. (rather than 5 a.m.) closing time resulted in an estimated 12.3 ± 2.4% reduction in total acute alcohol-related harms, a 7.9 ± 0.8% reduction in violence, an 11.9 ± 2.1% reduction in ED presentations and a 9.5 ± 1.8% reduction in hospitalizations. Further reductions were achieved simulating a 1 a.m. closing time, including a 17.5 ± 1.1% reduction in alcohol-related violence. Simulated extensions to bottle shop trading hours resulted in increases in rates of all four measures of harm, although most of the effects came from increasing operating hours from 10 p.m. to 11 p.m. CONCLUSIONS: An agent-based simulation model suggests that restricting trading hours of licensed venues reduces rates of alcohol-related harm and extending trading hours of bottle shops increases rates of alcohol-related harm. The model can estimate the effects of a range of policy options.

Static metrics of impact for a dynamic problem: The need for smarter tools to guide suicide prevention planning and investment.

OBJECTIVES: This study investigates two approaches to estimate the potential impact of a population-level intervention on Australian suicide, to highlight the importance of selecting appropriate analytic approaches for informing evidence-based strategies for suicide prevention. METHODS: The potential impact of a psychosocial therapy intervention on the incidence of suicide in Australia over the next 10 years was used as a case study to compare the potential impact on suicides averted using: (1) a traditional epidemiological measure of population attributable risk and (2) a dynamic measure of population impact based on a systems science model of suicide that incorporates changes over time. RESULTS: Based on the population preventive fraction, findings suggest that the psychosocial therapy intervention if implemented among all eligible individuals in the Australian population would prevent 5.4% of suicides (or 1936 suicides) over the next 10 years. In comparison, estimates from the dynamic simulation model which accounts for changes in the effect size of the intervention over time, the time taken for the intervention to have an impact in the population, and likely barriers to the uptake and availability of services suggest that the intervention would avert a lower proportion of suicides (between 0.4% and 0.5%) over the same follow-up period. CONCLUSION: Traditional epidemiological measures used to estimate population health burden have several limitations that are often understated and can lead to unrealistic expectations of the potential impact of evidence-based interventions in real-world settings. This study highlights these limitations and proposes an alternative analytic approach to guide policy and practice decisions to achieve reductions in Australian suicide.

Improving the quality of healthcare: a cross-sectional study of the features of successful clinical networks.

OBJECTIVES: Networks of clinical experts are being established internationally to help embed evidence based care in health systems. There is emerging evidence that these clinical networks can drive quality improvement programs, but the features that distinguish successful networks are largely unknown. We examined the factors that make clinical networks effective at improving quality of care and facilitating system-wide changes. METHODS: We conducted a retrospective cross-sectional study of 19 state-wide clinical networks that reflected a range of medical and surgical specialty care and were in operation from 2006 to 2008 in New South Wales, Australia. We conducted qualitative interviews with network leaders to characterise potential impacts, and conducted internet surveys of network members to evaluate external support and the organisational and program characteristics of their respective networks. The main outcome measures were median ratings of individual network impacts on quality of care and system-wide changes, determined through independent assessment of documented evidence by an expert panel. RESULTS: We interviewed 19 network managers and 32 network co-chairs; 592 network members completed internet surveys. Three networks were rated as having had high impact on quality of care, and seven as having had high impact on system-wide change. Better-perceived strategic and operational network management was significantly associated with higher ratings of impact on quality of care (coefficient estimate 0.86; 95% confidence interval [CI] 0.02, 1.69). Better-perceived leadership of the network manager (coefficient estimate 0.47; 95% CI 0.10, 0.85) and strategic and operational network management (coefficient estimate 0.23; 95% CI 0.06, 0.41) were associated with higher ratings of impact on system-wide change. CONCLUSIONS: This study represents the largest study of clinical networks undertaken to date. The results suggest that clinical networks that span the health system can improve quality of care and facilitate system-wide change. Network management and leadership, encompassing both strategic and operational elements at the organisational level, appear to be the primary influences on network success. These findings can guide future organisational and system-wide change programs and the development or strengthening of clinical networks to help implement evidence based care to improve service delivery and outcomes.

Harnessing advances in computer simulation to inform policy and planning to reduce alcohol-related harms.

OBJECTIVES: Alcohol misuse is a complex systemic problem. The aim of this study was to explore the feasibility of using a transparent and participatory agent-based modelling approach to develop a robust decision support tool to test alcohol policy scenarios before they are implemented in the real world. METHODS: A consortium of Australia's leading alcohol experts was engaged to collaboratively develop an agent-based model of alcohol consumption behaviour and related harms. As a case study, four policy scenarios were examined. RESULTS: A 19.5 ± 2.5% reduction in acute alcohol-related harms was estimated with the implementation of a 3 a.m. licensed venue closing time plus 1 a.m. lockout; and a 9 ± 2.6% reduction in incidence was estimated with expansion of treatment services to reach 20% of heavy drinkers. Combining the two scenarios produced a 33.3 ± 2.7% reduction in the incidence of acute alcohol-related harms, suggesting a synergistic effect. CONCLUSIONS: This study demonstrates the feasibility of participatory development of a contextually relevant computer simulation model of alcohol-related harms and highlights the value of the approach in identifying potential policy responses that best leverage limited resources.

Knowledge mobilisation for policy development: implementing systems approaches through participatory dynamic simulation modelling.

BACKGROUND: Evidence-based decision-making is an important foundation for health policy and service planning decisions, yet there remain challenges in ensuring that the many forms of available evidence are considered when decisions are being made. Mobilising knowledge for policy and practice is an emergent process, and one that is highly relational, often messy and profoundly context dependent. Systems approaches, such as dynamic simulation modelling can be used to examine both complex health issues and the context in which they are embedded, and to develop decision support tools. OBJECTIVE: This paper reports on the novel use of participatory simulation modelling as a knowledge mobilisation tool in Australian real-world policy settings. We describe how this approach combined systems science methodology and some of the core elements of knowledge mobilisation best practice. We describe the strategies adopted in three case studies to address both technical and socio-political issues, and compile the experiential lessons derived. Finally, we consider the implications of these knowledge mobilisation case studies and provide evidence for the feasibility of this approach in policy development settings. CONCLUSION: Participatory dynamic simulation modelling builds on contemporary knowledge mobilisation approaches for health stakeholders to collaborate and explore policy and health service scenarios for priority public health topics. The participatory methods place the decision-maker at the centre of the process and embed deliberative methods and co-production of knowledge. The simulation models function as health policy and programme dynamic decision support tools that integrate diverse forms of evidence, including research evidence, expert knowledge and localised contextual information. Further research is underway to determine the impact of these methods on health service decision-making.

A decision-support tool to inform Australian strategies for preventing suicide and suicidal behaviour.

Dynamic simulation modelling is increasingly being recognised as a valuable decision-support tool to help guide investments and actions to address complex public health issues such as suicide. In particular, participatory system dynamics (SD) modelling provides a useful tool for asking high-level 'what if' questions, and testing the likely impacts of different combinations of policies and interventions at an aggregate level before they are implemented in the real world. We developed an SD model for suicide prevention in Australia, and investigated the hypothesised impacts over the next 10 years (2015-2025) of a combination of current intervention strategies proposed for population interventions in Australia: 1) general practitioner (GP) training, 2) coordinated aftercare in those who have attempted suicide, 3) school-based mental health literacy programs, 4) brief-contact interventions in hospital settings, and 5) psychosocial treatment approaches. Findings suggest that the largest reductions in suicide were associated with GP training (6%) and coordinated aftercare approaches (4%), with total reductions of 12% for all interventions combined. This paper highlights the value of dynamic modelling methods for managing complexity and uncertainty, and demonstrates their potential use as a decision-support tool for policy makers and program planners for community suicide prevention actions.

Participatory simulation modelling to inform public health policy and practice: Rethinking the evidence hierarchies.

Drawing on the long tradition of evidence-based medicine that aims to improve the efficiency and effectiveness of clinical practice, the field of public health has sought to apply 'hierarchies of evidence' to appraise and synthesise public health research. Various critiques of this approach led to the development of synthesis methods that include broader evidence typologies and more 'fit for purpose' privileging of methodological designs. While such adaptations offer great utility for evidence-informed public health policy and practice, this paper offers an alternative perspective on the synthesis of evidence that necessitates a yet more egalitarian approach. Dynamic simulation modelling is increasingly recognised as a valuable evidence synthesis tool to inform public health policy and programme planning for complex problems. The development of simulation models draws on and privileges a wide range of evidence typologies, thus challenging the traditional use of 'hierarchies of evidence' to support decisions on complex dynamic problems.