On Sequential Bayesian Inference for Continual Learning.

Sequential Bayesian inference can be used for continual learning to prevent catastrophic forgetting of past tasks and provide an informative prior when learning new tasks. We revisit sequential Bayesian inference and assess whether using the previous task's posterior as a prior for a new task can prevent catastrophic forgetting in Bayesian neural networks. Our first contribution is to perform sequential Bayesian inference using Hamiltonian Monte Carlo. We propagate the posterior as a prior for new tasks by approximating the posterior via fitting a density estimator on Hamiltonian Monte Carlo samples. We find that this approach fails to prevent catastrophic forgetting, demonstrating the difficulty in performing sequential Bayesian inference in neural networks. From there, we study simple analytical examples of sequential Bayesian inference and CL and highlight the issue of model misspecification, which can lead to sub-optimal continual learning performance despite exact inference. Furthermore, we discuss how task data imbalances can cause forgetting. From these limitations, we argue that we need probabilistic models of the continual learning generative process rather than relying on sequential Bayesian inference over Bayesian neural network weights. Our final contribution is to propose a simple baseline called Prototypical Bayesian Continual Learning, which is competitive with the best performing Bayesian continual learning methods on class incremental continual learning computer vision benchmarks.

Coolsense® versus EMLA® for peripheral venous cannulation in adult volunteers: A randomised crossover trial.

Peripheral venous cannulation (PVC) is a commonly performed invasive medical procedure. Topical treatments such as the eutectic mixture of local anaesthetics (EMLA®, Aspen Pharmacare Australia Pty Ltd, St Leonards, NSW) attenuate the associated pain, but are limited by requiring up to one hour of application before becoming effective. The Coolsense® (Coolsense Medical Ltd., Tel Aviv, Israel) pain numbing applicator is a new device using a cryoanalgesic means to anaesthetise skin within seconds. Coolsense is being increasingly used for cannulation, but comparative studies are lacking. We recruited 64 healthy adult volunteers to this open-label two sequence, two period randomised crossover trial. Participants had two 20 gauge venous cannulae inserted, one on the dorsum of each hand. Each cannulation attempt was preceded by treatment with Coolsense or an EMLA patch containing 2.5% lidocaine and 2.5% prilocaine. The primary outcome was participant pain using the 0-10 numerical pain rating scale. Secondary outcomes were participant satisfaction scores on a 0-10 scale, treatment preference, and failed cannulation attempts. Participants were randomly assigned to either the Coolsense EMLA (n = 32) or EMLA Coolsense (n = 32) sequence. All participants completed the trial. The pooled mean paired difference of the numerical pain rating scale was -1.84 (95% confidence intervals -1.28 to -2.41; P < 0.001) in favour of EMLA. The pooled mean paired difference for satisfaction score was 2.26 (95% confidence intervals 1.46 to 3.07; P < 0.001) higher with EMLA. Most participants preferred EMLA over Coolsense (P < 0.001). There was no significant difference regarding failed cannulation between the two treatments (P = 0.14). Among healthy individuals undergoing elective PVC, EMLA was associated with reduced pain, increased satisfaction, and was the preferred treatment compared to Coolsense.

Variational integrator graph networks for learning energy-conserving dynamical systems.

Recent advances show that neural networks embedded with physics-informed priors significantly outperform vanilla neural networks in learning and predicting the long-term dynamics of complex physical systems from noisy data. Despite this success, there has only been a limited study on how to optimally combine physics priors to improve predictive performance. To tackle this problem we unpack and generalize recent innovations into individual inductive bias segments. As such, we are able to systematically investigate all possible combinations of inductive biases of which existing methods are a natural subset. Using this framework we introduce variational integrator graph networks-a novel method that unifies the strengths of existing approaches by combining an energy constraint, high-order symplectic variational integrators, and graph neural networks. We demonstrate, across an extensive ablation, that the proposed unifying framework outperforms existing methods, for data-efficient learning and in predictive accuracy, across both single- and many-body problems studied in the recent literature. We empirically show that the improvements arise because high-order variational integrators combined with a potential energy constraint induce coupled learning of generalized position and momentum updates which can be formalized via the partitioned Runge-Kutta method.

Port-Hamiltonian neural networks for learning explicit time-dependent dynamical systems.

Accurately learning the temporal behavior of dynamical systems requires models with well-chosen learning biases. Recent innovations embed the Hamiltonian and Lagrangian formalisms into neural networks and demonstrate a significant improvement over other approaches in predicting trajectories of physical systems. These methods generally tackle autonomous systems that depend implicitly on time or systems for which a control signal is known a priori. Despite this success, many real world dynamical systems are nonautonomous, driven by time-dependent forces and experience energy dissipation. In this study, we address the challenge of learning from such nonautonomous systems by embedding the port-Hamiltonian formalism into neural networks, a versatile framework that can capture energy dissipation and time-dependent control forces. We show that the proposed port-Hamiltonian neural network can efficiently learn the dynamics of nonlinear physical systems of practical interest and accurately recover the underlying stationary Hamiltonian, time-dependent force, and dissipative coefficient. A promising outcome of our network is its ability to learn and predict chaotic systems such as the Duffing equation, for which the trajectories are typically hard to learn.

Synthesis, characterisation and water-gas shift activity of nano-particulate mixed-metal (Al, Ti) cobalt oxides.

The formation of mixed-metal cobalt oxides, representing potential metal-support compounds for cobalt-based catalysts, has been observed at high conversion levels in the Fischer-Tropsch synthesis over metal oxide-supported cobalt catalysts. An often observed increase in the carbon dioxide selectivity at Fischer-Tropsch conversion levels above 80% has been suggested to be associated to the formation of water-gas shift active oxidic cobalt species. Mixed-metal cobalt oxides, namely cobalt aluminate and cobalt titanate, were therefore synthesised and tested for potential catalytic activity in the water-gas shift reaction. We present a preparation route for amorphous mixed-metal oxides via thermal treatment of metal precursors in benzyl alcohol. Calcination of the as prepared nanoparticles results in highly crystalline phases. The nano-particulate mixed-metal cobalt oxides were thoroughly analysed by means of X-ray diffraction, Raman spectroscopy, temperature-programmed reduction, X-ray absorption near edge structure spectroscopy, extended X-ray absorption fine structure, and high-resolution scanning transmission electron microscopy. This complementary characterisation of the synthesised materials allows for a distinct identification of the phases and their properties. The cobalt aluminate prepared has a cobalt-rich composition (Co1+xAl2-xO4) with a homogeneous atomic distribution throughout the nano-particulate structures, while the perovskite-type cobalt titanate (CoTiO3) features cobalt-lean smaller particles associated with larger ones with an increased concentration of cobalt. The cobalt aluminate prepared showed no water-gas shift activity in the medium-shift temperature range, while the cobalt titanate sample catalysed the conversion of water and carbon monoxide to hydrogen and carbon dioxide after an extended activation period. However, this perovskite underwent vast restructuring forming metallic cobalt, a known catalyst for the water-gas shift reaction at temperatures exceeding typical conditions for the cobalt-based Fischer-Tropsch synthesis, and anatase-TiO2. The partial reduction of the mixed-metal oxide and segregation was identified by means of post-run characterisation using X-ray diffraction, Raman spectroscopy, and transmission electron microscopy energy-dispersive spectrometry.

Gaussian Processes for Personalized Interpretable Volatility Metrics in the Step-Down Ward.

Patients in a hospital step-down unit require a level of care that is between that of the intensive care unit (ICU) and that of the general ward. While many patients remain physiologically stabilized, others will suffer clinical emergencies and be readmitted to the ICU, with a subsequent high risk of mortality. Had the associated physiological deterioration been detected early, the emergency may have been less severe or avoided entirely. Current clinical monitoring is largely heuristic, requiring manual calculation of risk scores and the use of heuristic decision criteria. Technical drawbacks include ignoring the time-series dynamics of physiological measurements, and lacking patient-specificity (i.e., personalization of models to the individual patient). In this paper, we demonstrate how Gaussian process regression models can supplement current monitoring practice by providing interpretable and intuitive illustrations of erratic vital-sign volatility. These personalized volatility metrics may provide significantly advanced warning of deterioration, while minimizing the false alarms that induce so-called alarm fatigue. While many AI-based approaches to healthcare are criticized for being uninterpretable "black-box" methods, the cause of alarms generated from the proposed methods are explicitly interpretable and intuitive. We conclude that intelligent computational inference using methods such as those proposed can enhance current clinical decision making and potentially save lives.

Salt-Enrichment Impact on Biomass Production in a Natural Population of Peatland Dwelling Arcellinida and Euglyphida (Testate Amoebae).

Unicellular free-living microbial eukaryotes of the order Arcellinida (Tubulinea; Amoebozoa) and Euglyphida (Cercozoa; SAR), commonly termed testate amoebae, colonise almost every freshwater ecosystem on Earth. Patterns in the distribution and productivity of these organisms are strongly linked to abiotic conditions-particularly moisture availability and temperature-however, the ecological impacts of changes in salinity remain poorly documented. Here, we examine how variable salt concentrations affect a natural community of Arcellinida and Euglyphida on a freshwater sub-Antarctic peatland. We principally report that deposition of wind-blown oceanic salt-spray aerosols onto the peatland surface corresponds to a strong reduction in biomass and to an alteration in the taxonomic composition of communities in favour of generalist taxa. Our results suggest novel applications of this response as a sensitive tool to monitor salinisation of coastal soils and to detect salinity changes within peatland palaeoclimate archives. Specifically, we suggest that these relationships could be used to reconstruct millennial scale variability in salt-spray deposition-a proxy for changes in wind-conditions-from sub-fossil communities of Arcellinida and Euglyphida preserved in exposed coastal peatlands.

Functional and informatics analysis enables glycosyltransferase activity prediction.

The elucidation and prediction of how changes in a protein result in altered activities and selectivities remain a major challenge in chemistry. Two hurdles have prevented accurate family-wide models: obtaining (i) diverse datasets and (ii) suitable parameter frameworks that encapsulate activities in large sets. Here, we show that a relatively small but broad activity dataset is sufficient to train algorithms for functional prediction over the entire glycosyltransferase superfamily 1 (GT1) of the plant Arabidopsis thaliana. Whereas sequence analysis alone failed for GT1 substrate utilization patterns, our chemical-bioinformatic model, GT-Predict, succeeded by coupling physicochemical features with isozyme-recognition patterns over the family. GT-Predict identified GT1 biocatalysts for novel substrates and enabled functional annotation of uncharacterized GT1s. Finally, analyses of GT-Predict decision pathways revealed structural modulators of substrate recognition, thus providing information on mechanisms. This multifaceted approach to enzyme prediction may guide the streamlined utilization (and design) of biocatalysts and the discovery of other family-wide protein functions.

Bayesian Optimization of Personalized Models for Patient Vital-Sign Monitoring.

Gaussian process regression (GPR) provides a means to generate flexible personalized models of time series of patient vital signs. These models can perform useful clinical inference in ways that population-based models cannot. A challenge for the use of personalized models is that they must be amenable to a wide range of parameterizations, to accommodate the plausible physiology of any patient in the population. Additionally, optimal performance is typically achieved when models are regularized in light of the knowledge of the physiology of the individual patient. In this paper, we describe a method to build GP models with varying complexity (via covariance kernels) and regularization (via fixed priors over hyperparameters) on a patient-specific level, for the purpose of robust vital-sign forecasting. To this end, our results present evidence in support of two main hypotheses: 1) the use of patient-specific models can outperform population-based models for useful clinical tasks, such as vital-sign forecasting; and 2) the optimal values of (hyper)parameters of these models are best determined by sophisticated methods of optimization, due to high correlation between dimensions of the search space. The resulting models are sufficiently robust to inform clinicians of a patient's vital-sign trajectory and warn of imminent deterioration.

Likelihood-based artefact detection in continuously-acquired patient vital signs.

Robust continuous monitoring of patient vital signs (VS) is limited by artefactual data yielding measurements that are not representative of the patient's physiology. These artefacts are typified by several distinct "archetypes". We present several of these archetypal artefacts for heart rate (HR) monitoring, and propose a light weight, real-time algorithm to remove the majority of these artefacts. Most artefacts are not identifiable by their values in absolute terms, but instead by their values relative to other measurements nearby in time. We model temporally-proximate measurements as independent and identically distributed (i.i.d.) samples from a Gamma distribution. Measurements with low likelihood with respect to the distribution are candidates for artefact removal. This lightweight algorithm is important for real-time deployment on wearable sensors, which are becoming increasingly common in hospital and home care. The clinical applicability of artefact-removal is demonstrated in its ability to enhance patient deterioration detection. A Kalman filter-based patient monitoring algorithm is shown to improve early warning of deterioration when the proposed artefact-removal algorithm is used. We demonstrate this real-time system with patient data from a clinical trial that we have undertaken.

West Antarctic Ice Sheet retreat driven by Holocene warm water incursions.

Glaciological and oceanographic observations coupled with numerical models show that warm Circumpolar Deep Water (CDW) incursions onto the West Antarctic continental shelf cause melting of the undersides of floating ice shelves. Because these ice shelves buttress glaciers feeding into them, their ocean-induced thinning is driving Antarctic ice-sheet retreat today. Here we present a multi-proxy data based reconstruction of variability in CDW inflow to the Amundsen Sea sector, the most vulnerable part of the West Antarctic Ice Sheet, during the Holocene epoch (from 11.7 thousand years ago to the present). The chemical compositions of foraminifer shells and benthic foraminifer assemblages in marine sediments indicate that enhanced CDW upwelling, controlled by the latitudinal position of the Southern Hemisphere westerly winds, forced deglaciation of this sector from at least 10,400 years ago until 7,500 years ago-when an ice-shelf collapse may have caused rapid ice-sheet thinning further upstream-and since the 1940s. These results increase confidence in the predictive capability of current ice-sheet models.

Past penguin colony responses to explosive volcanism on the Antarctic Peninsula.

Changes in penguin populations on the Antarctic Peninsula have been linked to several environmental factors, but the potentially devastating impact of volcanic activity has not been considered. Here we use detailed biogeochemical analyses to track past penguin colony change over the last 8,500 years on Ardley Island, home to one of the Antarctic Peninsula's largest breeding populations of gentoo penguins. The first sustained penguin colony was established on Ardley Island c. 6,700 years ago, pre-dating sub-fossil evidence of Peninsula-wide occupation by c. 1,000 years. The colony experienced five population maxima during the Holocene. Overall, we find no consistent relationships with local-regional atmospheric and ocean temperatures or sea-ice conditions, although the colony population maximum, c. 4,000-3,000 years ago, corresponds with regionally elevated temperatures. Instead, at least three of the five phases of penguin colony expansion were abruptly ended by large eruptions from the Deception Island volcano, resulting in near-complete local extinction of the colony, with, on average, 400-800 years required for sustainable recovery.

Monitoring chicken flock behaviour provides early warning of infection by human pathogen Campylobacter.

Campylobacter is the commonest bacterial cause of gastrointestinal infection in humans, and chicken meat is the major source of infection throughout the world. Strict and expensive on-farm biosecurity measures have been largely unsuccessful in controlling infection and are hampered by the time needed to analyse faecal samples, with the result that Campylobacter status is often known only after a flock has been processed. Our data demonstrate an alternative approach that monitors the behaviour of live chickens with cameras and analyses the 'optical flow' patterns made by flock movements. Campylobacter-free chicken flocks have higher mean and lower kurtosis of optical flow than those testing positive for Campylobacter by microbiological methods. We show that by monitoring behaviour in this way, flocks likely to become positive can be identified within the first 7-10 days of life, much earlier than conventional on-farm microbiological methods. This early warning has the potential to lead to a more targeted approach to Campylobacter control and also provides new insights into possible sources of infection that could transform the control of this globally important food-borne pathogen.

Bayesian Gaussian processes for identifying the deteriorating patient.

The step-down unit (SDU) is a high-acuity hospital environment, to which patients may be sent after discharge from the intensive care unit (ICU). About 1- in-7 patients will deteriorate in the SDU and require emergency readmission to the ICU. Upon readmission, these patients experience significantly higher mortality risks and lengths of stay. Gaussian process regression (GPR) models are proposed as a flexible, principled, probabilistic method to address the clinical need to monitor continuously patient time-series of vital signs acquired in the SDU. The proposed GPR models focus on the robust forecasting of patient heart rate time-series and on the early detection of patient deterioration. The proposed methods are tested with an SDU data set from the University of Pittsburgh Medical Center, comprising 333 patients, 59 of whom had at least one verified clinical emergency event. Results suggest that GPR-based heart rate monitoring provides superior advanced warning of deterioration compared to the current clinical practice of rules-based thresholding, and slightly outperforms the current state-of-the-art kernel density method, which requires 4 additional vital sign features.

Antarctic crabs: invasion or endurance?

Recent scientific interest following the "discovery" of lithodid crabs around Antarctica has centred on a hypothesis that these crabs might be poised to invade the Antarctic shelf if the recent warming trend continues, potentially decimating its native fauna. This "invasion hypothesis" suggests that decapod crabs were driven out of Antarctica 40-15 million years ago and are only now returning as "warm" enough habitats become available. The hypothesis is based on a geographically and spatially poor fossil record of a different group of crabs (Brachyura), and examination of relatively few Recent lithodid samples from the Antarctic slope. In this paper, we examine the existing lithodid fossil record and present the distribution and biogeographic patterns derived from over 16,000 records of Recent Southern Hemisphere crabs and lobsters. Globally, the lithodid fossil record consists of only two known specimens, neither of which comes from the Antarctic. Recent records show that 22 species of crabs and lobsters have been reported from the Southern Ocean, with 12 species found south of 60 °S. All are restricted to waters warmer than 0 °C, with their Antarctic distribution limited to the areas of seafloor dominated by Circumpolar Deep Water (CDW). Currently, CDW extends further and shallower onto the West Antarctic shelf than the known distribution ranges of most lithodid species examined. Geological evidence suggests that West Antarctic shelf could have been available for colonisation during the last 9,000 years. Distribution patterns, species richness, and levels of endemism all suggest that, rather than becoming extinct and recently re-invading from outside Antarctica, the lithodid crabs have likely persisted, and even radiated, on or near to Antarctic slope. We conclude there is no evidence for a modern-day "crab invasion". We recommend a repeated targeted lithodid sampling program along the West Antarctic shelf to fully test the validity of the "invasion hypothesis".

Prediction of welfare outcomes for broiler chickens using Bayesian regression on continuous optical flow data.

Currently, assessment of broiler (meat) chicken welfare relies largely on labour-intensive or post-mortem measures of welfare. We here describe a method for continuously and robustly monitoring the welfare of living birds while husbandry changes are still possible. We detail the application of Bayesian modelling to motion data derived from the output of cameras placed in commercial broiler houses. We show that the forecasts produced by the model can be used to accurately assess certain key aspects of the future health and welfare of a flock. The difference between healthy flocks and less-healthy ones becomes predictable days or even weeks before clinical symptoms become apparent. Hockburn (damaged leg skin, usually only seen in birds of two weeks or older) can be well predicted in flocks of only 1-2 days of age, using this approach. Our model combines optical flow descriptors of bird motion with robust multivariate forecasting and provides a sparse, efficient model with sparsity-inducing priors to achieve maximum predictive power with the minimum number of key variables.

Inferring social network structure in ecological systems from spatio-temporal data streams.

We propose a methodology for extracting social network structure from spatio-temporal datasets that describe timestamped occurrences of individuals. Our approach identifies temporal regions of dense agent activity and links are drawn between individuals based on their co-occurrences across these 'gathering events'. The statistical significance of these connections is then tested against an appropriate null model. Such a framework allows us to exploit the wealth of analytical and computational tools of network analysis in settings where the underlying connectivity pattern between interacting agents (commonly termed the adjacency matrix) is not given a priori. We perform experiments on two large-scale datasets (greater than 10(6) points) of great tit Parus major wild bird foraging records and illustrate the use of this approach by examining the temporal dynamics of pairing behaviour, a process that was previously very hard to observe. We show that established pair bonds are maintained continuously, whereas new pair bonds form at variable times before breeding, but are characterized by a rapid development of network proximity. The method proposed here is general, and can be applied to any system with information about the temporal co-occurrence of interacting agents.

Bayesian inference for an adaptive Ordered Probit model: an application to Brain Computer Interfacing.

This paper proposes an algorithm for adaptive, sequential classification in systems with unknown labeling errors, focusing on the biomedical application of Brain Computer Interfacing (BCI). The method is shown to be robust in the presence of label and sensor noise. We focus on the inference and prediction of target labels under a nonlinear and non-Gaussian model. In order to handle missing or erroneous labeling, we model observed labels as a noisy observation of a latent label set with multiple classes (≥ 2). Whilst this paper focuses on the method's application to BCI systems, the algorithm has the potential to be applied to many application domains in which sequential missing labels are to be imputed in the presence of uncertainty. This dynamic classification algorithm combines an Ordered Probit model and an Extended Kalman Filter (EKF). The EKF estimates the parameters of the Ordered Probit model sequentially with time. We test the performance of the classification approach by processing synthetic datasets and real experimental EEG signals with multiple classes (2, 3 and 4 labels) for a Brain Computer Interfacing (BCI) experiment.

Prediction of feather damage in laying hens using optical flows and Markov models.

Feather pecking in laying hens is a major welfare and production problem for commercial egg producers, resulting in mortality, loss of production as well as welfare issues for the damaged birds. Damaging outbreaks of feather pecking are currently impossible to control, despite a number of proposed interventions. However, the ability to predict feather damage in advance would be a valuable research tool for identifying which management or environmental factors could be the most effective interventions at different ages. This paper proposes a framework for forecasting the damage caused by injurious pecking based on automated image processing and statistical analysis. By frame-by-frame analysis of video recordings of laying hen flocks, optical flow measures are calculated as indicators of the movement of the birds. From the optical flow datasets, measures of disturbance are extracted using hidden Markov models. Based on these disturbance measures and age-related variables, the levels of feather damage in flocks in future weeks is predicted. Applying the proposed method to real-world datasets, it is shown that the disturbance measures offer improved predictive values for feather damage thus enabling an identification of flocks with probable prevalence of damage and injury later in lay.