Reinforcement learning for decision-making under deep uncertainty.

Planning under complex uncertainty often asks for plans that can adapt to changing future conditions. To inform plan development during this process, exploration methods have been used to explore the performance of candidate policies given uncertainties. Nevertheless, these methods hardly enable adaptation by themselves, so extra efforts are required to develop the final adaptive plans, hence compromising the overall decision-making efficiency. This paper introduces Reinforcement Learning (RL) that employs closed-loop control as a new exploration method that enables automated adaptive policy-making for planning under uncertainty. To investigate its performance, we compare RL with a widely-used exploration method, Multi-Objective Evolutionary Algorithm (MOEA), in two hypothetical problems via computational experiments. Our results indicate the complementarity of the two methods. RL makes better use of its exploration history, hence always providing higher efficiency and providing better policy robustness in the presence of parameter uncertainty. MOEA quantifies objective uncertainty in a more intuitive way, hence providing better robustness to objective uncertainty. These findings will help researchers choose appropriate methods in different applications.

The enabling institutional context for integrated water management: lessons from Melbourne.

There is widespread international acceptance that climate change, demographic shifts and resource limitations impact on the performance of water servicing in cities. In response to these challenges, many scholars propose that a fundamental move away from traditional centralised infrastructure towards more integrated water management is required. However, there is limited practical or scholarly understanding of how to enable this change in practice and few modern cities have done so successfully. This paper addresses this gap by analysing empirical evidence of Melbourne's recent experience in shifting towards a hybrid of centralised and decentralised infrastructure to draw lessons about the institutional context that enabled this shift. The research was based on a qualitative single-case study, involving interviews and envisioning workshops with urban water practitioners who have been directly involved in Melbourne's water system changes. It was found that significant changes occurred in the cultural-cognitive, normative and regulative dimensions of Melbourne's water system. These included a shift in cultural beliefs for the water profession, new knowledge through evidence and learning, additional water servicing goals and priorities, political leadership, community pressure, better coordinated governance arrangements and strong market mechanisms. The paper synthesises lessons from the case study that, with further development, could form the basis of prescriptive guidance for enabling the shift to new modes of water servicing to support more liveable, sustainable and resilient outcomes for future cities.

A socio-technical model to explore urban water systems scenarios.

This article reports on the ongoing work and research involved in the development of a socio-technical model of urban water systems. Socio-technical means the model is not so much concerned with the technical or biophysical aspects of urban water systems, but rather with the social and institutional implications of the urban water infrastructure and vice versa. A socio-technical model, in the view purported in this article, produces scenarios of different urban water servicing solutions gaining or losing influence in meeting water-related societal needs, like potable water, drainage, environmental health and amenity. The urban water system is parameterised with vectors of the relative influence of each servicing solution. The model is a software implementation of the Multi-Pattern Approach, a theory on societal systems, like urban water systems, and how these develop and go through transitions under various internal and external conditions. Acknowledging that social dynamics comes with severe and non-reducible uncertainties, the model is set up to be exploratory, meaning that for any initial condition several possible future scenarios are produced. This article gives a concise overview of the necessary theoretical background, the model architecture and some initial test results using a drainage example.