FLAMINGO: calibrating large cosmological hydrodynamical simulations with machine learning.

To fully take advantage of the data provided by large-scale structure surveys, we need to quantify the potential impact of baryonic effects, such as feedback from active galactic nuclei (AGN) and star formation, on cosmological observables. In simulations, feedback processes originate on scales that remain unresolved. Therefore, they need to be sourced via subgrid models that contain free parameters. We use machine learning to calibrate the AGN and stellar feedback models for the FLAMINGO (Fullhydro Large-scale structure simulations with All-sky Mapping for the Interpretation of Next Generation Observations) cosmological hydrodynamical simulations. Using Gaussian process emulators trained on Latin hypercubes of 32 smaller volume simulations, we model how the galaxy stellar mass function (SMF) and cluster gas fractions change as a function of the subgrid parameters. The emulators are then fit to observational data, allowing for the inclusion of potential observational biases. We apply our method to the three different FLAMINGO resolutions, spanning a factor of 64 in particle mass, recovering the observed relations within the respective resolved mass ranges. We also use the emulators, which link changes in subgrid parameters to changes in observables, to find models that skirt or exceed the observationally allowed range for cluster gas fractions and the SMF. Our method enables us to define model variations in terms of the data that they are calibrated to rather than the values of specific subgrid parameters. This approach is useful, because subgrid parameters are typically not directly linked to particular observables, and predictions for a specific observable are influenced by multiple subgrid parameters.

Demonstrating multi-country calibration of a tuberculosis model using new history matching and emulation package - hmer.

Infectious disease models are widely used by epidemiologists to improve the understanding of transmission dynamics and disease natural history, and to predict the possible effects of interventions. As the complexity of such models increases, however, it becomes increasingly challenging to robustly calibrate them to empirical data. History matching with emulation is a calibration method that has been successfully applied to such models, but has not been widely used in epidemiology partly due to the lack of available software. To address this issue, we developed a new, user-friendly R package hmer to simply and efficiently perform history matching with emulation. In this paper, we demonstrate the first use of hmer for calibrating a complex deterministic model for the country-level implementation of tuberculosis vaccines to 115 low- and middle-income countries. The model was fit to 9-13 target measures, by varying 19-22 input parameters. Overall, 105 countries were successfully calibrated. Among the remaining countries, hmer visualisation tools, combined with derivative emulation methods, provided strong evidence that the models were misspecified and could not be calibrated to the target ranges. This work shows that hmer can be used to simply and rapidly calibrate a complex model to data from over 100 countries, making it a useful addition to the epidemiologist's calibration tool-kit.

Ab initio predictions link the neutron skin of 208Pb to nuclear forces.

Heavy atomic nuclei have an excess of neutrons over protons, which leads to the formation of a neutron skin whose thickness is sensitive to details of the nuclear force. This links atomic nuclei to properties of neutron stars, thereby relating objects that differ in size by orders of magnitude. The nucleus 208Pb is of particular interest because it exhibits a simple structure and is experimentally accessible. However, computing such a heavy nucleus has been out of reach for ab initio theory. By combining advances in quantum many-body methods, statistical tools and emulator technology, we make quantitative predictions for the properties of 208Pb starting from nuclear forces that are consistent with symmetries of low-energy quantum chromodynamics. We explore 109 different nuclear force parameterizations via history matching, confront them with data in select light nuclei and arrive at an importance-weighted ensemble of interactions. We accurately reproduce bulk properties of 208Pb and determine the neutron skin thickness, which is smaller and more precise than a recent extraction from parity-violating electron scattering but in agreement with other experimental probes. This work demonstrates how realistic two- and three-nucleon forces act in a heavy nucleus and allows us to make quantitative predictions across the nuclear landscape.

Complex model calibration through emulation, a worked example for a stochastic epidemic model.

Uncertainty quantification is a formal paradigm of statistical estimation that aims to account for all uncertainties inherent in the modelling process of real-world complex systems. The methods are directly applicable to stochastic models in epidemiology, however they have thus far not been widely used in this context. In this paper, we provide a tutorial on uncertainty quantification of stochastic epidemic models, aiming to facilitate the use of the uncertainty quantification paradigm for practitioners with other complex stochastic simulators of applied systems. We provide a formal workflow including the important decisions and considerations that need to be taken, and illustrate the methods over a simple stochastic epidemic model of UK SARS-CoV-2 transmission and patient outcome. We also present new approaches to visualisation of outputs from sensitivity analyses and uncertainty quantification more generally in high input and/or output dimensions.

Challenges in estimation, uncertainty quantification and elicitation for pandemic modelling.

The estimation of parameters and model structure for informing infectious disease response has become a focal point of the recent pandemic. However, it has also highlighted a plethora of challenges remaining in the fast and robust extraction of information using data and models to help inform policy. In this paper, we identify and discuss four broad challenges in the estimation paradigm relating to infectious disease modelling, namely the Uncertainty Quantification framework, data challenges in estimation, model-based inference and prediction, and expert judgement. We also postulate priorities in estimation methodology to facilitate preparation for future pandemics.

June: open-source individual-based epidemiology simulation.

We introduce June, an open-source framework for the detailed simulation of epidemics on the basis of social interactions in a virtual population constructed from geographically granular census data, reflecting age, sex, ethnicity and socio-economic indicators. Interactions between individuals are modelled in groups of various sizes and properties, such as households, schools and workplaces, and other social activities using social mixing matrices. June provides a suite of flexible parametrizations that describe infectious diseases, how they are transmitted and affect contaminated individuals. In this paper, we apply June to the specific case of modelling the spread of COVID-19 in England. We discuss the quality of initial model outputs which reproduce reported hospital admission and mortality statistics at national and regional levels as well as by age strata.

Understanding hormonal crosstalk in Arabidopsis root development via emulation and history matching.

A major challenge in plant developmental biology is to understand how plant growth is coordinated by interacting hormones and genes. To meet this challenge, it is important to not only use experimental data, but also formulate a mathematical model. For the mathematical model to best describe the true biological system, it is necessary to understand the parameter space of the model, along with the links between the model, the parameter space and experimental observations. We develop sequential history matching methodology, using Bayesian emulation, to gain substantial insight into biological model parameter spaces. This is achieved by finding sets of acceptable parameters in accordance with successive sets of physical observations. These methods are then applied to a complex hormonal crosstalk model for Arabidopsis root growth. In this application, we demonstrate how an initial set of 22 observed trends reduce the volume of the set of acceptable inputs to a proportion of 6.1 × 10-7 of the original space. Additional sets of biologically relevant experimental data, each of size 5, reduce the size of this space by a further three and two orders of magnitude respectively. Hence, we provide insight into the constraints placed upon the model structure by, and the biological consequences of, measuring subsets of observations.

Predicting badger visits to farm yards and making predictions available to farmers.

The use of agricultural resources or environments by wildlife may result in opportunities for transmission of infections amongst wild animals, livestock and humans. Targeted use of biosecurity measures may therefore reduce disease risks, although this requires practical knowledge of where such measures would be most effective, and effective means of communicating risks so that stakeholders can make informed decisions about such investment. In parts of Europe, the European badger Meles meles may act as a wildlife reservoir for Mycobacterium bovis, the causative agent of bovine tuberculosis, and badger visits to farmyards may provide potential opportunities for transmission of M. bovis to cattle. Biosecurity measures are effective in reducing badger activity in farmyards, although it is unclear which farms should be targeted with such measures. We used cameras to monitor badger activity in 155 farmyards in south west England and Wales, and related variations in the presence and frequency of badger visits to farm characteristics. Badgers were recorded on camera in 40% of farmyards monitored. However, the frequency of visits was highly variable, with badgers recorded on >50% of nights in only 10% of farms. The presence of badgers in farmyards was positively associated with the density of badger setts, the number of feed stores and the number of cattle sheds, and negatively associated with the distance to the nearest active badger sett, the presence of a house/dwelling and the number of cattle housed on the farm. The frequency of visits was negatively associated with the distance to the nearest active badger sett and the number of cattle housed. Models predicted the presence/absence of badgers in farmyards with 73% accuracy (62% sensitivity, 81% specificity, using a cut off value of 0.265). Models could not distinguish between farms with low/high frequency of visits, although farms predicted as having badgers present typically had a higher frequency of visits than those that were not. We developed and present an interactive web based application: the Badger Farm Assessment Tool (BFAT), to allow users to enter the characteristics of a farm and generate a relative risk score describing the likelihood of badger visits.

Choice of time horizon critical in estimating costs and effects of changes to HIV programmes.

BACKGROUND: Uganda changed its antiretroviral therapy guidelines in 2014, increasing the CD4 threshold for antiretroviral therapy initiation from 350 cells/µl to 500 cells/µl. We investigate what effect this change in policy is likely to have on HIV incidence, morbidity, and programme costs, and estimate the cost-effectiveness of the change over different time horizons. METHODS: We used a complex individual-based model of HIV transmission and antiretroviral therapy scale-up in Uganda. 100 model fits were generated by fitting the model to 51 demographic, sexual behaviour, and epidemiological calibration targets, varying 96 input parameters, using history matching with model emulation. An additional 19 cost and disability weight parameters were varied during the analysis of the model results. For each model fit, the model was run to 2030, with and without the change in threshold to 500 cells/µl. RESULTS: The change in threshold led to a 9.7% (90% plausible range: 4.3%-15.0%) reduction in incidence in 2030, and averted 278,944 (118,452-502,790) DALYs, at a total cost of $28M (-$142M to +$195M). The cost per disability adjusted life year (DALY) averted fell over time, from $3238 (-$125 to +$29,969) in 2014 to $100 (-$499 to +$785) in 2030. The change in threshold was cost-effective (cost <3×Uganda's per capita GDP per DALY averted) by 2018, and highly cost-effective (cost

Bayesian uncertainty analysis for complex systems biology models: emulation, global parameter searches and evaluation of gene functions.

BACKGROUND: Many mathematical models have now been employed across every area of systems biology. These models increasingly involve large numbers of unknown parameters, have complex structure which can result in substantial evaluation time relative to the needs of the analysis, and need to be compared to observed data of various forms. The correct analysis of such models usually requires a global parameter search, over a high dimensional parameter space, that incorporates and respects the most important sources of uncertainty. This can be an extremely difficult task, but it is essential for any meaningful inference or prediction to be made about any biological system. It hence represents a fundamental challenge for the whole of systems biology. METHODS: Bayesian statistical methodology for the uncertainty analysis of complex models is introduced, which is designed to address the high dimensional global parameter search problem. Bayesian emulators that mimic the systems biology model but which are extremely fast to evaluate are embeded within an iterative history match: an efficient method to search high dimensional spaces within a more formal statistical setting, while incorporating major sources of uncertainty. RESULTS: The approach is demonstrated via application to a model of hormonal crosstalk in Arabidopsis root development, which has 32 rate parameters, for which we identify the sets of rate parameter values that lead to acceptable matches between model output and observed trend data. The multiple insights into the model's structure that this analysis provides are discussed. The methodology is applied to a second related model, and the biological consequences of the resulting comparison, including the evaluation of gene functions, are described. CONCLUSIONS: Bayesian uncertainty analysis for complex models using both emulators and history matching is shown to be a powerful technique that can greatly aid the study of a large class of systems biology models. It both provides insight into model behaviour and identifies the sets of rate parameters of interest.

Improving ART programme retention and viral suppression are key to maximising impact of treatment as prevention - a modelling study.

BACKGROUND: UNAIDS calls for fewer than 500,000 new HIV infections/year by 2020, with treatment-as-prevention being a key part of their strategy for achieving the target. A better understanding of the contribution to transmission of people at different stages of the care pathway can help focus intervention services at populations where they may have the greatest effect. We investigate this using Uganda as a case study. METHODS: An individual-based HIV/ART model was fitted using history matching. 100 model fits were generated to account for uncertainties in sexual behaviour, HIV epidemiology, and ART coverage up to 2015 in Uganda. A number of different ART scale-up intervention scenarios were simulated between 2016 and 2030. The incidence and proportion of transmission over time from people with primary infection, post-primary ART-naïve infection, and people currently or previously on ART was calculated. RESULTS: In all scenarios, the proportion of transmission by ART-naïve people decreases, from 70% (61%-79%) in 2015 to between 23% (15%-40%) and 47% (35%-61%) in 2030. The proportion of transmission by people on ART increases from 7.8% (3.5%-13%) to between 14% (7.0%-24%) and 38% (21%-55%). The proportion of transmission by ART dropouts increases from 22% (15%-33%) to between 31% (23%-43%) and 56% (43%-70%). CONCLUSIONS: People who are currently or previously on ART are likely to play an increasingly large role in transmission as ART coverage increases in Uganda. Improving retention on ART, and ensuring that people on ART remain virally suppressed, will be key in reducing HIV incidence in Uganda.

Universal test, treat, and keep: improving ART retention is key in cost-effective HIV control in Uganda.

BACKGROUND: With ambitious new UNAIDS targets to end AIDS by 2030, and new WHO treatment guidelines, there is increased interest in the best way to scale-up ART coverage. We investigate the cost-effectiveness of various ART scale-up options in Uganda. METHODS: Individual-based HIV/ART model of Uganda, calibrated using history matching. 22 ART scale-up strategies were simulated from 2016 to 2030, comprising different combinations of six single interventions (1. increased HIV testing rates, 2. no CD4 threshold for ART initiation, 3. improved ART retention, 4. increased ART restart rates, 5. improved linkage to care, 6. improved pre-ART care). The incremental net monetary benefit (NMB) of each intervention was calculated, for a wide range of different willingness/ability to pay (WTP) per DALY averted (health-service perspective, 3% discount rate). RESULTS: For all WTP thresholds above $210, interventions including removing the CD4 threshold were likely to be most cost-effective. At a WTP of $715 (1 × per-capita-GDP) interventions to improve linkage to and retention/re-enrolment in HIV care were highly likely to be more cost-effective than interventions to increase rates of HIV testing. At higher WTP (> ~ $1690), the most cost-effective option was 'Universal Test, Treat, and Keep' (UTTK), which combines interventions 1-5 detailed above. CONCLUSIONS: Our results support new WHO guidelines to remove the CD4 threshold for ART initiation in Uganda. With additional resources, this could be supplemented with interventions aimed at improving linkage to and/or retention in HIV care. To achieve the greatest reductions in HIV incidence, a UTTK policy should be implemented.

Bayesian history matching of complex infectious disease models using emulation: a tutorial and a case study on HIV in Uganda.

Advances in scientific computing have allowed the development of complex models that are being routinely applied to problems in disease epidemiology, public health and decision making. The utility of these models depends in part on how well they can reproduce empirical data. However, fitting such models to real world data is greatly hindered both by large numbers of input and output parameters, and by long run times, such that many modelling studies lack a formal calibration methodology. We present a novel method that has the potential to improve the calibration of complex infectious disease models (hereafter called simulators). We present this in the form of a tutorial and a case study where we history match a dynamic, event-driven, individual-based stochastic HIV simulator, using extensive demographic, behavioural and epidemiological data available from Uganda. The tutorial describes history matching and emulation. History matching is an iterative procedure that reduces the simulator's input space by identifying and discarding areas that are unlikely to provide a good match to the empirical data. History matching relies on the computational efficiency of a Bayesian representation of the simulator, known as an emulator. Emulators mimic the simulator's behaviour, but are often several orders of magnitude faster to evaluate. In the case study, we use a 22 input simulator, fitting its 18 outputs simultaneously. After 9 iterations of history matching, a non-implausible region of the simulator input space was identified that was 10(11) times smaller than the original input space. Simulator evaluations made within this region were found to have a 65% probability of fitting all 18 outputs. History matching and emulation are useful additions to the toolbox of infectious disease modellers. Further research is required to explicitly address the stochastic nature of the simulator as well as to account for correlations between outputs.