Modeling COVID-19 disease processes by remote elicitation of causal Bayesian networks from medical experts.

BACKGROUND: COVID-19 is a new multi-organ disease causing considerable worldwide morbidity and mortality. While many recognized pathophysiological mechanisms are involved, their exact causal relationships remain opaque. Better understanding is needed for predicting their progression, targeting therapeutic approaches, and improving patient outcomes. While many mathematical causal models describe COVID-19 epidemiology, none have described its pathophysiology. METHODS: In early 2020, we began developing such causal models. The SARS-CoV-2 virus's rapid and extensive spread made this particularly difficult: no large patient datasets were publicly available; the medical literature was flooded with sometimes conflicting pre-review reports; and clinicians in many countries had little time for academic consultations. We used Bayesian network (BN) models, which provide powerful calculation tools and directed acyclic graphs (DAGs) as comprehensible causal maps. Hence, they can incorporate both expert opinion and numerical data, and produce explainable, updatable results. To obtain the DAGs, we used extensive expert elicitation (exploiting Australia's exceptionally low COVID-19 burden) in structured online sessions. Groups of clinical and other specialists were enlisted to filter, interpret and discuss the literature and develop a current consensus. We encouraged inclusion of theoretically salient latent (unobservable) variables, likely mechanisms by extrapolation from other diseases, and documented supporting literature while noting controversies. Our method was iterative and incremental: systematically refining and validating the group output using one-on-one follow-up meetings with original and new experts. 35 experts contributed 126 hours face-to-face, and could review our products. RESULTS: We present two key models, for the initial infection of the respiratory tract and the possible progression to complications, as causal DAGs and BNs with corresponding verbal descriptions, dictionaries and sources. These are the first published causal models of COVID-19 pathophysiology. CONCLUSIONS: Our method demonstrates an improved procedure for developing BNs via expert elicitation, which other teams can implement to model emergent complex phenomena. Our results have three anticipated applications: (i) freely disseminating updatable expert knowledge; (ii) guiding design and analysis of observational and clinical studies; (iii) developing and validating automated tools for causal reasoning and decision support. We are developing such tools for the initial diagnosis, resource management, and prognosis of COVID-19, parameterized using the ISARIC and LEOSS databases.

Predicting postpartum haemorrhage: A systematic review of prognostic models.

BACKGROUND: Postpartum haemorrhage (PPH) remains a leading cause of maternal mortality and morbidity worldwide, and the rate is increasing. Using a reliable predictive model could identify those at risk, support management and treatment, and improve maternal outcomes. AIMS: To systematically identify and appraise existing prognostic models for PPH and ascertain suitability for clinical use. MATERIALS AND METHODS: MEDLINE, CINAHL, Embase, and the Cochrane Library were searched using combinations of terms and synonyms, including 'postpartum haemorrhage', 'prognostic model', and 'risk factors'. Observational or experimental studies describing a prognostic model for risk of PPH, published in English, were included. The Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies checklist informed data extraction and the Prediction Model Risk of Bias Assessment Tool guided analysis. RESULTS: Sixteen studies met the inclusion criteria after screening 1612 records. All studies were hospital settings from eight different countries. Models were developed for women who experienced vaginal birth (n = 7), caesarean birth (n = 2), any type of birth (n = 2), hypertensive disorders (n = 1) and those with placental abnormalities (n = 4). All studies were at high risk of bias due to use of inappropriate analysis methods or omission of important statistical considerations or suboptimal validation. CONCLUSIONS: No existing prognostic models for PPH are ready for clinical application. Future research is needed to externally validate existing models and potentially develop a new model that is reliable and applicable to clinical practice.

BARD: A Structured Technique for Group Elicitation of Bayesian Networks to Support Analytic Reasoning.

In many complex, real-world situations, problem solving and decision making require effective reasoning about causation and uncertainty. However, human reasoning in these cases is prone to confusion and error. Bayesian networks (BNs) are an artificial intelligence technology that models uncertain situations, supporting better probabilistic and causal reasoning and decision making. However, to date, BN methodologies and software require (but do not include) substantial upfront training, do not provide much guidance on either the model building process or on using the model for reasoning and reporting, and provide no support for building BNs collaboratively. Here, we contribute a detailed description and motivation for our new methodology and application, Bayesian ARgumentation via Delphi (BARD). BARD utilizes BNs and addresses these shortcomings by integrating (1) short, high-quality e-courses, tips, and help on demand; (2) a stepwise, iterative, and incremental BN construction process; (3) report templates and an automated explanation tool; and (4) a multiuser web-based software platform and Delphi-style social processes. The result is an end-to-end online platform, with associated online training, for groups without prior BN expertise to understand and analyze a problem, build a model of its underlying probabilistic causal structure, validate and reason with the causal model, and (optionally) use it to produce a written analytic report. Initial experiments demonstrate that, for suitable problems, BARD aids in reasoning and reporting. Comparing their effect sizes also suggests BARD's BN-building and collaboration combine beneficially and cumulatively.

Individuals vs. BARD: Experimental Evaluation of an Online System for Structured, Collaborative Bayesian Reasoning.

US intelligence analysts must weigh up relevant evidence to assess the probability of their conclusions, and express this reasoning clearly in written reports for decision-makers. Typically, they work alone with no special analytic tools, and sometimes succumb to common probabilistic and causal reasoning errors. So, the US government funded a major research program (CREATE) for four large academic teams to develop new structured, collaborative, software-based methods that might achieve better results. Our team's method (BARD) is the first to combine two key techniques: constructing causal Bayesian network models (BNs) to represent analyst knowledge, and small-group collaboration via the Delphi technique. BARD also incorporates compressed, high-quality online training allowing novices to use it, and checklist-inspired report templates with a rudimentary AI tool for generating text explanations from analysts' BNs. In two prior experiments, our team showed BARD's BN-building assists probabilistic reasoning when used by individuals, with a large effect (Glass' Δ 0.8) (Cruz et al., 2020), and even minimal Delphi-style interactions improve the BN structures individuals produce, with medium to very large effects (Glass' Δ 0.5-1.3) (Bolger et al., 2020). This experiment is the critical test of BARD as an integrated system and possible alternative to business-as-usual for intelligence analysis. Participants were asked to solve three probabilistic reasoning problems spread over 5 weeks, developed by our team to test both quantitative accuracy and susceptibility to tempting qualitative fallacies. Our 256 participants were randomly assigned to form 25 teams of 6-9 using BARD and 58 individuals using Google Suite and (if desired) the best pen-and-paper techniques. For each problem, BARD outperformed this control with very large to huge effects (Glass' Δ 1.4-2.2), greatly exceeding CREATE's initial target. We conclude that, for suitable problems, BARD already offers significant advantages over both business-as-usual and existing BN software. Our effect sizes also suggest BARD's BN-building and collaboration combined beneficially and cumulatively, although implementation differences decreased performances compared to Cruz et al. (2020), so interaction may have contributed. BARD has enormous potential for further development and testing of specific components and on more complex problems, and many potential applications beyond intelligence analysis.

Verbal probabilities: Very likely to be somewhat more confusing than numbers.

People interpret verbal expressions of probabilities (e.g. 'very likely') in different ways, yet words are commonly preferred to numbers when communicating uncertainty. Simply providing numerical translations alongside reports or text containing verbal probabilities should encourage consistency, but these guidelines are often ignored. In an online experiment with 924 participants, we compared four different formats for presenting verbal probabilities with the numerical guidelines used in the US Intelligence Community Directive (ICD) 203 to see whether any could improve the correspondence between the intended meaning and participants' interpretation ('in-context'). This extends previous work in the domain of climate science. The four experimental conditions we tested were: 1. numerical guidelines bracketed in text, e.g. X is very unlikely (05-20%), 2. click to see the full guidelines table in a new window, 3. numerical guidelines appear in a mouse over tool tip, and 4. no guidelines provided (control). Results indicate that correspondence with the ICD 203 standard is substantially improved only when numerical guidelines are bracketed in text. For this condition, average correspondence was 66%, compared with 32% in the control. We also elicited 'context-free' numerical judgements from participants for each of the seven verbal probability expressions contained in ICD 203 (i.e., we asked participants what range of numbers they, personally, would assign to those expressions), and constructed 'evidence-based lexicons' based on two methods from similar research, 'membership functions' and 'peak values', that reflect our large sample's intuitive translations of the terms. Better aligning the intended and assumed meaning of fuzzy words like 'unlikely' can reduce communication problems between the reporter and receiver of probabilistic information. In turn, this can improve decision making under uncertainty.

HetFHMM: A Novel Approach to Infer Tumor Heterogeneity Using Factorial Hidden Markov Models.

Cancer arises from successive rounds of mutations, resulting in tumor cells with different somatic mutations known as clones. Drug responsiveness and therapeutics of cancer depend on the accurate detection of clones in a tumor sample. Recent research has considered inferring clonal composition of a tumor sample using computational models based on short read data of the sample generated using next-generation sequencing (NGS) technology. Short reads (segmented DNA parts of different tumor cells) are noisy; therefore, inferring the clones and their mutations from the data is a difficult and complex problem. We develop a new model called HetFHMM, based on factorial hidden Markov models, to infer clones and their proportions from noisy NGS data. In our model, each hidden chain represents the genomic signature of a clone, and a mixture of chains results in the observed data. We make use of Gibbs sampling and exponentiated gradient algorithms to infer the hidden variables and mixing proportions. We compare our model with strong models from previous work (PyClone and PhyloSub) based on both synthetic data and real cancer data on acute myeloid leukemia. Empirical results confirm that HetFHMM infers clonal composition of a tumor sample more accurately than previous work.

Bayesian networks for clinical decision support in lung cancer care.

Survival prediction and treatment selection in lung cancer care are characterised by high levels of uncertainty. Bayesian Networks (BNs), which naturally reason with uncertain domain knowledge, can be applied to aid lung cancer experts by providing personalised survival estimates and treatment selection recommendations. Based on the English Lung Cancer Database (LUCADA), we evaluate the feasibility of BNs for these two tasks, while comparing the performances of various causal discovery approaches to uncover the most feasible network structure from expert knowledge and data. We show first that the BN structure elicited from clinicians achieves a disappointing area under the ROC curve of 0.75 (± 0.03), whereas a structure learned by the CAMML hybrid causal discovery algorithm, which adheres with the temporal restrictions, achieves 0.81 (± 0.03). Second, our causal intervention results reveal that BN treatment recommendations, based on prescribing the treatment plan that maximises survival, can only predict the recorded treatment plan 29% of the time. However, this percentage rises to 76% when partial matches are included.

Incorporating expert knowledge when learning Bayesian network structure: a medical case study.

OBJECTIVES: Bayesian networks (BNs) are rapidly becoming a leading technology in applied Artificial Intelligence, with many applications in medicine. Both automated learning of BNs and expert elicitation have been used to build these networks, but the potentially more useful combination of these two methods remains underexplored. In this paper we examine a number of approaches to their combination when learning structure and present new techniques for assessing their results. METHODS AND MATERIALS: Using public-domain medical data, we run an automated causal discovery system, CaMML, which allows the incorporation of multiple kinds of prior expert knowledge into its search, to test and compare unbiased discovery with discovery biased with different kinds of expert opinion. We use adjacency matrices enhanced with numerical and colour labels to assist with the interpretation of the results. We present an algorithm for generating a single BN from a set of learned BNs that incorporates user preferences regarding complexity vs completeness. These techniques are presented as part of the first detailed workflow for hybrid structure learning within the broader knowledge engineering process. RESULTS: The detailed knowledge engineering workflow is shown to be useful for structuring a complex iterative BN development process. The adjacency matrices make it clear that for our medical case study using the IOWA dataset, the simplest kind of prior information (partially sorting variables into tiers) was more effective in aiding model discovery than either using no prior information or using more sophisticated and detailed expert priors. The method for generating a single BN captures relationships that would be overlooked by other approaches in the literature. CONCLUSION: Hybrid causal learning of BNs is an important emerging technology. We present methods for incorporating it into the knowledge engineering process, including visualisation and analysis of the learned networks.