Complex model calibration through emulation, a worked example for a stochastic epidemic model.

Uncertainty quantification is a formal paradigm of statistical estimation that aims to account for all uncertainties inherent in the modelling process of real-world complex systems. The methods are directly applicable to stochastic models in epidemiology, however they have thus far not been widely used in this context. In this paper, we provide a tutorial on uncertainty quantification of stochastic epidemic models, aiming to facilitate the use of the uncertainty quantification paradigm for practitioners with other complex stochastic simulators of applied systems. We provide a formal workflow including the important decisions and considerations that need to be taken, and illustrate the methods over a simple stochastic epidemic model of UK SARS-CoV-2 transmission and patient outcome. We also present new approaches to visualisation of outputs from sensitivity analyses and uncertainty quantification more generally in high input and/or output dimensions.

Challenges in estimation, uncertainty quantification and elicitation for pandemic modelling.

The estimation of parameters and model structure for informing infectious disease response has become a focal point of the recent pandemic. However, it has also highlighted a plethora of challenges remaining in the fast and robust extraction of information using data and models to help inform policy. In this paper, we identify and discuss four broad challenges in the estimation paradigm relating to infectious disease modelling, namely the Uncertainty Quantification framework, data challenges in estimation, model-based inference and prediction, and expert judgement. We also postulate priorities in estimation methodology to facilitate preparation for future pandemics.

Challenges on the interaction of models and policy for pandemic control.

The COVID-19 pandemic has seen infectious disease modelling at the forefront of government decision-making. Models have been widely used throughout the pandemic to estimate pathogen spread and explore the potential impact of different intervention strategies. Infectious disease modellers and policymakers have worked effectively together, but there are many avenues for progress on this interface. In this paper, we identify and discuss seven broad challenges on the interaction of models and policy for pandemic control. We then conclude with suggestions and recommendations for the future.

The importance of uncertainty quantification in model reproducibility.

Many computer models possess high-dimensional input spaces and substantial computational time to produce a single model evaluation. Although such models are often 'deterministic', these models suffer from a wide range of uncertainties. We argue that uncertainty quantification is crucial for computer model validation and reproducibility. We present a statistical framework, termed history matching, for performing global parameter search by comparing model output to the observed data. We employ Gaussian process (GP) emulators to produce fast predictions about model behaviour at the arbitrary input parameter settings allowing output uncertainty distributions to be calculated. History matching identifies sets of input parameters that give rise to acceptable matches between observed data and model output given our representation of uncertainties. Modellers could proceed by simulating computer models' outputs of interest at these identified parameter settings and producing a range of predictions. The variability in model results is crucial for inter-model comparison as well as model development. We illustrate the performance of emulation and history matching on a simple one-dimensional toy model and in application to a climate model. This article is part of the theme issue 'Reliability and reproducibility in computational science: implementing verification, validation and uncertainty quantification in silico'.

Key questions for modelling COVID-19 exit strategies.

Combinations of intense non-pharmaceutical interventions (lockdowns) were introduced worldwide to reduce SARS-CoV-2 transmission. Many governments have begun to implement exit strategies that relax restrictions while attempting to control the risk of a surge in cases. Mathematical modelling has played a central role in guiding interventions, but the challenge of designing optimal exit strategies in the face of ongoing transmission is unprecedented. Here, we report discussions from the Isaac Newton Institute 'Models for an exit strategy' workshop (11-15 May 2020). A diverse community of modellers who are providing evidence to governments worldwide were asked to identify the main questions that, if answered, would allow for more accurate predictions of the effects of different exit strategies. Based on these questions, we propose a roadmap to facilitate the development of reliable models to guide exit strategies. This roadmap requires a global collaborative effort from the scientific community and policymakers, and has three parts: (i) improve estimation of key epidemiological parameters; (ii) understand sources of heterogeneity in populations; and (iii) focus on requirements for data collection, particularly in low-to-middle-income countries. This will provide important information for planning exit strategies that balance socio-economic benefits with public health.

Uncertainty and variability in computational and mathematical models of cardiac physiology.

KEY POINTS: Mathematical and computational models of cardiac physiology have been an integral component of cardiac electrophysiology since its inception, and are collectively known as the Cardiac Physiome. We identify and classify the numerous sources of variability and uncertainty in model formulation, parameters and other inputs that arise from both natural variation in experimental data and lack of knowledge. The impact of uncertainty on the outputs of Cardiac Physiome models is not well understood, and this limits their utility as clinical tools. We argue that incorporating variability and uncertainty should be a high priority for the future of the Cardiac Physiome. We suggest investigating the adoption of approaches developed in other areas of science and engineering while recognising unique challenges for the Cardiac Physiome; it is likely that novel methods will be necessary that require engagement with the mathematics and statistics community. ABSTRACT: The Cardiac Physiome effort is one of the most mature and successful applications of mathematical and computational modelling for describing and advancing the understanding of physiology. After five decades of development, physiological cardiac models are poised to realise the promise of translational research via clinical applications such as drug development and patient-specific approaches as well as ablation, cardiac resynchronisation and contractility modulation therapies. For models to be included as a vital component of the decision process in safety-critical applications, rigorous assessment of model credibility will be required. This White Paper describes one aspect of this process by identifying and classifying sources of variability and uncertainty in models as well as their implications for the application and development of cardiac models. We stress the need to understand and quantify the sources of variability and uncertainty in model inputs, and the impact of model structure and complexity and their consequences for predictive model outputs. We propose that the future of the Cardiac Physiome should include a probabilistic approach to quantify the relationship of variability and uncertainty of model inputs and outputs.

Risk analysis for autonomous underwater vehicle operations in extreme environments.

Autonomous underwater vehicles (AUVs) are used increasingly to explore hazardous marine environments. Risk assessment for such complex systems is based on subjective judgment and expert knowledge as much as on hard statistics. Here, we describe the use of a risk management process tailored to AUV operations, the implementation of which requires the elicitation of expert judgment. We conducted a formal judgment elicitation process where eight world experts in AUV design and operation were asked to assign a probability of AUV loss given the emergence of each fault or incident from the vehicle's life history of 63 faults and incidents. After discussing methods of aggregation and analysis, we show how the aggregated risk estimates obtained from the expert judgments were used to create a risk model. To estimate AUV survival with mission distance, we adopted a statistical survival function based on the nonparametric Kaplan-Meier estimator. We present theoretical formulations for the estimator, its variance, and confidence limits. We also present a numerical example where the approach is applied to estimate the probability that the Autosub3 AUV would survive a set of missions under Pine Island Glacier, Antarctica in January-March 2009.

A soft solid surface on Titan as revealed by the Huygens Surface Science Package.

The surface of Saturn's largest satellite--Titan--is largely obscured by an optically thick atmospheric haze, and so its nature has been the subject of considerable speculation and discussion. The Huygens probe entered Titan's atmosphere on 14 January 2005 and descended to the surface using a parachute system. Here we report measurements made just above and on the surface of Titan by the Huygens Surface Science Package. Acoustic sounding over the last 90 m above the surface reveals a relatively smooth, but not completely flat, surface surrounding the landing site. Penetrometry and accelerometry measurements during the probe impact event reveal that the surface was neither hard (like solid ice) nor very compressible (like a blanket of fluffy aerosol); rather, the Huygens probe landed on a relatively soft solid surface whose properties are analogous to wet clay, lightly packed snow and wet or dry sand. The probe settled gradually by a few millimetres after landing.

Measurements of the offshore wave climate around the British Isles by satellite altimeter.

Satellite altimetry gives a new perspective on ocean wave climate. Measurements around the British Isles show a strong seasonality, with exceptionally large average wave heights to the west and north of the British Isles in the winter. Furthermore, the interannual variability of winter wave climate is very high. Most of this variability can be described by a strong linear dependence on the North Atlantic Oscillation (NAO) index. This relationship may largely explain observations of increasing wave heights in the northeastern Atlantic and northern North Sea during the latter decades of the 20th century, coincident with a long-term rise in the NAO.

Assimilating satellite ocean-colour observations into oceanic ecosystem models.

The effectiveness of ocean-colour data assimilation in providing robust biological-parameter estimates for basin-scale ecosystem models is investigated for a phytoplankton-zooplankton-nutrient model using North Atlantic satellite chlorophyll data. The model is forced by annual cycles of mixed-layer depth, day length, photosynthetically available radiation and a temperature-dependent phytoplankton maximum growth rate. Although ocean-colour data are potentially limited in their ability to constrain model parameters because they provide information about the phytoplankton component only, this limitation is offset by the volume of data available covering the range of possible biogeochemical responses to similar and widely varying physical conditions. The results are improved by applying wintertime nutrient estimates based on in situ observations as an additional constraint. Repeatability of parameter estimates obtained from independent samples is examined. Results obtained using regional and basin-wide sampling strategies for obtaining the optimization dataset are compared and the geographic applicability of the calibrated models is assessed.

Rossby waves: synergy in action.

Rossby waves are an important phenomenon, linking processes in the west of ocean basins with forcing that occurred earlier in the east. We show evidence for such features in satellite-derived datasets of sea-surface height, temperature and ocean colour, using a section of the south Indian Ocean as an example. We discuss the possible mechanisms for an effect on chlorophyll, and we investigate this by comparing the ocean colour signal with the other datasets. In this region, the primary mechanism for a Rossby-wave signal in ocean colour appears to be meridional advection of water across a strong chlorophyll gradient. However, this cannot fully explain the observations in the westernmost basin.