Modeling COVID-19 disease processes by remote elicitation of causal Bayesian networks from medical experts.

BACKGROUND: COVID-19 is a new multi-organ disease causing considerable worldwide morbidity and mortality. While many recognized pathophysiological mechanisms are involved, their exact causal relationships remain opaque. Better understanding is needed for predicting their progression, targeting therapeutic approaches, and improving patient outcomes. While many mathematical causal models describe COVID-19 epidemiology, none have described its pathophysiology. METHODS: In early 2020, we began developing such causal models. The SARS-CoV-2 virus's rapid and extensive spread made this particularly difficult: no large patient datasets were publicly available; the medical literature was flooded with sometimes conflicting pre-review reports; and clinicians in many countries had little time for academic consultations. We used Bayesian network (BN) models, which provide powerful calculation tools and directed acyclic graphs (DAGs) as comprehensible causal maps. Hence, they can incorporate both expert opinion and numerical data, and produce explainable, updatable results. To obtain the DAGs, we used extensive expert elicitation (exploiting Australia's exceptionally low COVID-19 burden) in structured online sessions. Groups of clinical and other specialists were enlisted to filter, interpret and discuss the literature and develop a current consensus. We encouraged inclusion of theoretically salient latent (unobservable) variables, likely mechanisms by extrapolation from other diseases, and documented supporting literature while noting controversies. Our method was iterative and incremental: systematically refining and validating the group output using one-on-one follow-up meetings with original and new experts. 35 experts contributed 126 hours face-to-face, and could review our products. RESULTS: We present two key models, for the initial infection of the respiratory tract and the possible progression to complications, as causal DAGs and BNs with corresponding verbal descriptions, dictionaries and sources. These are the first published causal models of COVID-19 pathophysiology. CONCLUSIONS: Our method demonstrates an improved procedure for developing BNs via expert elicitation, which other teams can implement to model emergent complex phenomena. Our results have three anticipated applications: (i) freely disseminating updatable expert knowledge; (ii) guiding design and analysis of observational and clinical studies; (iii) developing and validating automated tools for causal reasoning and decision support. We are developing such tools for the initial diagnosis, resource management, and prognosis of COVID-19, parameterized using the ISARIC and LEOSS databases.

Predicting the causative pathogen among children with pneumonia using a causal Bayesian network.

BACKGROUND: Pneumonia remains a leading cause of hospitalization and death among young children worldwide, and the diagnostic challenge of differentiating bacterial from non-bacterial pneumonia is the main driver of antibiotic use for treating pneumonia in children. Causal Bayesian networks (BNs) serve as powerful tools for this problem as they provide clear maps of probabilistic relationships between variables and produce results in an explainable way by incorporating both domain expert knowledge and numerical data. METHODS: We used domain expert knowledge and data in combination and iteratively, to construct, parameterise and validate a causal BN to predict causative pathogens for childhood pneumonia. Expert knowledge elicitation occurred through a series of group workshops, surveys and one-on-one meetings involving 6-8 experts from diverse domain areas. The model performance was evaluated based on both quantitative metrics and qualitative expert validation. Sensitivity analyses were conducted to investigate how the target output is influenced by varying key assumptions of a particularly high degree of uncertainty around data or domain expert knowledge. RESULTS: Designed to apply to a cohort of children with X-ray confirmed pneumonia who presented to a tertiary paediatric hospital in Australia, the resulting BN offers explainable and quantitative predictions on a range of variables of interest, including the diagnosis of bacterial pneumonia, detection of respiratory pathogens in the nasopharynx, and the clinical phenotype of a pneumonia episode. Satisfactory numeric performance has been achieved including an area under the receiver operating characteristic curve of 0.8 in predicting clinically-confirmed bacterial pneumonia with sensitivity 88% and specificity 66% given certain input scenarios (i.e., information that is available and entered into the model) and trade-off preferences (i.e., relative weightings of the consequences of false positive versus false negative predictions). We specifically highlight that a desirable model output threshold for practical use is very dependent upon different input scenarios and trade-off preferences. Three commonly encountered scenarios were presented to demonstrate the potential usefulness of the BN outputs in various clinical pictures. CONCLUSIONS: To our knowledge, this is the first causal model developed to help determine the causative pathogen for paediatric pneumonia. We have shown how the method works and how it would help decision making on the use of antibiotics, providing insight into how computational model predictions may be translated to actionable decisions in practice. We discussed key next steps including external validation, adaptation and implementation. Our model framework and the methodological approach can be adapted beyond our context to broad respiratory infections and geographical and healthcare settings.

Urinary tract infections in children: building a causal model-based decision support tool for diagnosis with domain knowledge and prospective data.

BACKGROUND: Diagnosing urinary tract infections (UTIs) in children in the emergency department (ED) is challenging due to the variable clinical presentations and difficulties in obtaining a urine sample free from contamination. Clinicians need to weigh a range of observations to make timely diagnostic and management decisions, a difficult task to achieve without support due to the complex interactions among relevant factors. Directed acyclic graphs (DAG) and causal Bayesian networks (BN) offer a way to explicitly outline the underlying disease, contamination and diagnostic processes, and to further make quantitative inference on the event of interest thus serving as a tool for decision support. METHODS: We prospectively collected data on children present to ED with suspected UTIs. Through knowledge elicitation workshops and one-on-one meetings, a DAG was co-developed with clinical domain experts (the Expert DAG) to describe the causal relationships among variables relevant to paediatric UTIs. The Expert DAG was combined with prospective data and further domain knowledge to inform the development of an application-oriented BN (the Applied BN), designed to support the diagnosis of UTI. We assessed the performance of the Applied BN using quantitative and qualitative methods. RESULTS: We summarised patient background, clinical and laboratory characteristics of 431 episodes of suspected UTIs enrolled from May 2019 to November 2020. The Expert DAG was presented with a narrative description, elucidating how infection, specimen contamination and management pathways causally interact to form the complex picture of paediatric UTIs. Parameterised using prospective data and expert-elicited parameters, the Applied BN achieved an excellent and stable performance in predicting Escherichia coli culture results, with a mean area under the receiver operating characteristic curve of 0.86 and a mean log loss of 0.48 based on 10-fold cross-validation. The BN predictions were reviewed via a validation workshop, and we illustrate how they can be presented for decision support using three hypothetical clinical scenarios. CONCLUSION: Causal BNs created from both expert knowledge and data can integrate case-specific information to provide individual decision support during the diagnosis of paediatric UTIs in ED. The model aids the interpretation of culture results and the diagnosis of UTIs, promising the prospect of improved patient care and judicious use of antibiotics.

A flexible method for parameterizing ranked nodes in Bayesian networks using Beta distributions.

When novice modelers first attempt to build a Bayesian network, they are often impressed with the intuitive graphical structures that capture their causal understanding. This favorable impression evaporates on proceeding to parameterization. Conditional probability tables (CPT) require parameters for often hundreds of very similar scenarios and specifying them in the absence of data can be overwhelming. The problem is even more severe when eliciting parameters from experts with limited time. Often, there is local structure with fewer parameters that better describes the relationship. Such structures include the Noisy OR, decision trees, and equations. These work well for modelers, but can be an issue for experts and particularly groups of experts. An alternative approach is to elicit only a few CPT rows and interpolate the remainder. This is a promising approach, as it can handle unknown structures and multiple experts, but existing techniques can be limited. Here, we present a flexible approach called InterBeta for performing CPT interpolation with ordered nodes. In the simplest case, just two CPT rows are needed, but this can be easily augmented with further information. The basic approach assumes input independence, but allows dependencies to be reintroduced as required, and can also be combined with other local structures such as decision trees or equations, leaving the interpolator to fill in the gaps. We explain the InterBeta method, describe its capabilities and limitations and how it compares to similar approaches and show how it can trade-off elicitation effort against faithfully representing expert understanding.

Preferred health outcome states following treatment for pulmonary exacerbations of cystic fibrosis.

BACKGROUND: Treatment for pulmonary exacerbations of cystic fibrosis (CF) can produce a range of positive and negative outcomes. Understanding which of these outcomes are achievable and desirable to people affected by disease is critical to agreeing to goals of therapy and determining endpoints for trials. The relative importance of outcomes resulting from treatment of these episodes are not reported. We aimed to (i) quantify the relative importance of outcomes resulting from treatment for pulmonary exacerbations and (ii) develop patient and proxy carer-reported weighted outcome measures for use in adults and children, respectively. METHODS: A discrete choice experiment (DCE) survey was conducted. Participants were asked to make a series of hypothetical decisions about treatment for pulmonary exacerbations to assess how they make trade-offs between different attributes of health. Data were analysed using a conditional logistic regression model. The correlation coefficients from these data were rescaled to enable generation of a composite health outcome score between 0 and 100 (worst to best health state). RESULTS: 362 individuals participated (167 people with CF and 195 carers); of these, 206 completed the survey (56.9%). Most participants were female and resided in Australia. Difficult/painful breathing had the greatest impact on the preferred health state amongst people with CF and carers alike. Avoidance of gastrointestinal problems also heavily influenced decision-making. CONCLUSIONS: These data should be considered when making treatment decisions and determining endpoints for trials. Further research is recommended to quantify the preferences of children and to determine whether these align with those of their carer(s).

Balancing the Elicitation Burden and the Richness of Expert Input When Quantifying Discrete Bayesian Networks.

Structured expert judgment (SEJ) is a method for obtaining estimates of uncertain quantities from groups of experts in a structured way designed to minimize the pervasive cognitive frailties of unstructured approaches. When the number of quantities required is large, the burden on the groups of experts is heavy, and resource constraints may mean that eliciting all the quantities of interest is impossible. Partial elicitations can be complemented with imputation methods for the remaining, unelicited quantities. In the case where the quantities of interest are conditional probability distributions, the natural relationship between the quantities can be exploited to impute missing probabilities. Here we test the Bayesian intelligence interpolation method and its variations for Bayesian network conditional probability tables, called "InterBeta." We compare the various outputs of InterBeta on two cases where conditional probability tables were elicited from groups of experts. We show that interpolated values are in good agreement with experts' values and give guidance on how InterBeta could be used to good effect to reduce expert burden in SEJ exercises.

BARD: A Structured Technique for Group Elicitation of Bayesian Networks to Support Analytic Reasoning.

In many complex, real-world situations, problem solving and decision making require effective reasoning about causation and uncertainty. However, human reasoning in these cases is prone to confusion and error. Bayesian networks (BNs) are an artificial intelligence technology that models uncertain situations, supporting better probabilistic and causal reasoning and decision making. However, to date, BN methodologies and software require (but do not include) substantial upfront training, do not provide much guidance on either the model building process or on using the model for reasoning and reporting, and provide no support for building BNs collaboratively. Here, we contribute a detailed description and motivation for our new methodology and application, Bayesian ARgumentation via Delphi (BARD). BARD utilizes BNs and addresses these shortcomings by integrating (1) short, high-quality e-courses, tips, and help on demand; (2) a stepwise, iterative, and incremental BN construction process; (3) report templates and an automated explanation tool; and (4) a multiuser web-based software platform and Delphi-style social processes. The result is an end-to-end online platform, with associated online training, for groups without prior BN expertise to understand and analyze a problem, build a model of its underlying probabilistic causal structure, validate and reason with the causal model, and (optionally) use it to produce a written analytic report. Initial experiments demonstrate that, for suitable problems, BARD aids in reasoning and reporting. Comparing their effect sizes also suggests BARD's BN-building and collaboration combine beneficially and cumulatively.

An Integrated Biosecurity Risk Assessment Model (IBRAM) For Evaluating the Risk of Import Pathways for the Establishment of Invasive Species.

An important aspect of analyzing the risk of unwanted organisms establishing in an area is understanding the pathways by which they arrive. Evaluating the risks of these pathways requires use of data from multiple sources, which frequently are uncertain. To address the needs of agencies responsible for biosecurity operations, we present an Integrated Biosecurity Risk Assessment Model (IBRAM) for evaluating the risk of establishment and dispersal of invasive species along trade pathways. The IBRAM framework consists of multiple linked models which describe pest entry into the country, escape along trade pathways, initial dispersal into the environment, habitat suitability, probabilities of establishment and spread, and the consequences of these invasions. Bayesian networks (BN) are used extensively to model these processes. The model includes dynamic BN components and geographic data, resulting in distributions of output parameters over spatial and temporal axes. IBRAM is supported by a web-based tool that allows users to run the model on real-world pest examples and investigate the impact of alternative risk management scenarios, to explore the effect of various interventions and resource allocations. Two case studies are provided as examples of how IBRAM may be used: Queensland fruit fly (Bactrocera tryoni) (Diptera: Tephritidae) and brown marmorated stink bug (Halyomorpha halys) (Hemiptera: Pentatomidae) are unwanted organisms with the potential to invade Aotearoa New Zealand, and IBRAM has been influential in evaluating the efficacy of pathway management to mitigate the risk of their establishment in the country.

Novel method to select meaningful outcomes for evaluation in clinical trials.

BACKGROUND: A standardised framework for selecting outcomes for evaluation in trials has been proposed by the Core Outcome Measures in Effectiveness Trials working group. However, this method does not specify how to ensure that the outcomes that are selected are causally related to the disease and the health intervention being studied. Causal network diagrams may help researchers identify outcomes that are both clinically meaningful and likely to be causally dependent on the intervention, and endpoints that are, in turn, causally dependent on those outcomes. We aimed to (1) develop a generalisable method for selecting outcomes and endpoints in trials and (2) apply this method to select outcomes for evaluation in a trial investigating treatment strategies for pulmonary exacerbations of cystic fibrosis (CF). METHODS: We conducted a series of online surveys and workshops among people affected by CF. We used a modified Delphi approach to develop a consensus list of important outcomes. A workshop involving domain experts elicited how these outcomes were causally related to the underlying pathophysiological processes. Meaningful outcomes were prioritised based on the extent to which each outcome captured separate rather than common aspects of the underlying pathophysiological process. RESULTS: The 10 prioritised outcomes were: breathing difficulty/pain, sputum production/clearance, fatigue, appetite, pain (not related to breathing), motivation/demoralisation, fevers/night sweats, treatment burden, inability to meet personal goals and avoidance of gastrointestinal symptoms. CONCLUSIONS: This proposed method for selecting meaningful outcomes for evaluation in clinical trials may improve the value of research as a basis for clinical decisions.

Predicting the causative pathogen among children with osteomyelitis using Bayesian networks - improving antibiotic selection in clinical practice.

Infection of bone, osteomyelitis (OM), is a serious bacterial infection in children requiring urgent antibiotic therapy. While biological specimens are often obtained and cultured to guide antibiotic selection, culture results may take several days, are often falsely negative, and may be falsely positive because of contamination by non-causative bacteria. This poses a dilemma for clinicians when choosing the most suitable antibiotic. Selecting an antibiotic which is too narrow in spectrum risks treatment failure; selecting an antibiotic which is too broad risks toxicity and promotes antibiotic resistance. We have developed a Bayesian Network (BN) model that can be used to guide individually targeted antibiotic therapy at point-of-care, by predicting the most likely causative pathogen in children with OM and the antibiotic with optimal expected utility. The BN explicitly models the complex relationship between the unobserved infecting pathogen, observed culture results, and clinical and demographic variables, and integrates data with critical expert knowledge under a causal inference framework. Development of this tool resulted from a multidisciplinary approach, involving experts in infectious diseases, modelling, paediatrics, microbiology, computer science and statistics. The model-predicted prevalence of causative pathogens among children with osteomyelitis were 56 % for Staphylococcus aureus, 17 % for 'other' culturable bacteria (like Streptococcus pyogenes), and 27 % for bacterial pathogens that are not culturable using routine methods (like Kingella kingae). Log loss cross-validation suggests that the model performance is robust, with the best fit to culture results achieved when data and expert knowledge were combined during parameterisation. AUC values of 0.68 - 0.77 were achieved for predicting culture results of different types of specimens. BN-recommended antibiotics were rated optimal or adequate by experts in 82-98% of 81 cases sampled from the cohort. We have demonstrated the potential use of BNs in improving antibiotic selection for children with OM, which we believe to be generalisable in the development of a broader range of decision support tools. With appropriate validation, such tools might be effectively deployed for real-time clinical decision support, to promote a shift in clinical practice from generic to individually-targeted antibiotic therapy, and ultimately improve the management and outcomes for a range of serious bacterial infections.

Discrete choice experiment to evaluate preferences of patients with cystic fibrosis among alternative treatment-related health outcomes: a protocol.

INTRODUCTION: Clinical decision-making is a complex process. Patient preference information regarding desirable health states should inform treatment and is critical to agreeing on goals of therapy. Cystic fibrosis (CF) is a common, inheritable multisystem disorder for which the major manifestation is progressive, chronic lung disease. Intermittent pulmonary exacerbations are a hallmark of disease and these drive lung damage that results in premature death. We suspect that clinicians make assumptions, most likely implicit assumptions, about outcomes that are desired by patients who are treated for pulmonary exacerbations. The aim of this study is to identify and quantify the preferences of patients with cystic fibrosis regarding treatment outcomes. METHODS AND ANALYSIS: We will develop a discrete choice experiment (DCE) in collaboration with people with CF and their carers, and evaluate how patients make trade-offs between different aspects of health-related status when considering treatment options. ETHICS AND DISSEMINATION: Ethics approval for all aspects of this study was granted by the Western Australia Child and Adolescent Health Service Human Research Ethics Committee [RGS903]. Weighted preference information from the DCE will be used to develop a multiattribute utility instrument as a measure of treatment success in the upcoming Bayesian Evidence-Adaptive Trial to optimise management of CF. Dissemination of results will also occur through peer-reviewed publications and presentations to relevant stakeholders and research networks.

Inference in the Wild: A Framework for Human Situation Assessment and a Case Study of Air Combat.

Situation awareness is a key construct in human factors and arises from a process of situation assessment (SA). SA comprises the perception of information, its integration with existing knowledge, the search for new information, and the prediction of the future state of the world, including the consequences of planned actions. Causal models implemented as Bayesian networks (BNs) are attractive for modeling all of these processes within a single, unified framework. We elicited declarative knowledge from two Royal Australian Air Force (RAAF) fighter pilots about the information sources used in the identification (ID) of airborne entities and the causal relationships between these sources. This knowledge was represented in a BN (the declarative model) that was evaluated against the performance of 19 RAAF fighter pilots in a low-fidelity simulation. Pilot behavior was well predicted by a simple associative model (the behavioral model) with only three attributes of ID. Search for information by pilots was largely compensatory and was near-optimal with respect to the behavioral model. The average revision of beliefs in response to evidence was close to Bayesian, but there was substantial variability. Together, these results demonstrate the value of BNs for modeling human SA.

Incorporating expert knowledge when learning Bayesian network structure: a medical case study.

OBJECTIVES: Bayesian networks (BNs) are rapidly becoming a leading technology in applied Artificial Intelligence, with many applications in medicine. Both automated learning of BNs and expert elicitation have been used to build these networks, but the potentially more useful combination of these two methods remains underexplored. In this paper we examine a number of approaches to their combination when learning structure and present new techniques for assessing their results. METHODS AND MATERIALS: Using public-domain medical data, we run an automated causal discovery system, CaMML, which allows the incorporation of multiple kinds of prior expert knowledge into its search, to test and compare unbiased discovery with discovery biased with different kinds of expert opinion. We use adjacency matrices enhanced with numerical and colour labels to assist with the interpretation of the results. We present an algorithm for generating a single BN from a set of learned BNs that incorporates user preferences regarding complexity vs completeness. These techniques are presented as part of the first detailed workflow for hybrid structure learning within the broader knowledge engineering process. RESULTS: The detailed knowledge engineering workflow is shown to be useful for structuring a complex iterative BN development process. The adjacency matrices make it clear that for our medical case study using the IOWA dataset, the simplest kind of prior information (partially sorting variables into tiers) was more effective in aiding model discovery than either using no prior information or using more sophisticated and detailed expert priors. The method for generating a single BN captures relationships that would be overlooked by other approaches in the literature. CONCLUSION: Hybrid causal learning of BNs is an important emerging technology. We present methods for incorporating it into the knowledge engineering process, including visualisation and analysis of the learned networks.