Modeling COVID-19 disease processes by remote elicitation of causal Bayesian networks from medical experts.

BACKGROUND: COVID-19 is a new multi-organ disease causing considerable worldwide morbidity and mortality. While many recognized pathophysiological mechanisms are involved, their exact causal relationships remain opaque. Better understanding is needed for predicting their progression, targeting therapeutic approaches, and improving patient outcomes. While many mathematical causal models describe COVID-19 epidemiology, none have described its pathophysiology. METHODS: In early 2020, we began developing such causal models. The SARS-CoV-2 virus's rapid and extensive spread made this particularly difficult: no large patient datasets were publicly available; the medical literature was flooded with sometimes conflicting pre-review reports; and clinicians in many countries had little time for academic consultations. We used Bayesian network (BN) models, which provide powerful calculation tools and directed acyclic graphs (DAGs) as comprehensible causal maps. Hence, they can incorporate both expert opinion and numerical data, and produce explainable, updatable results. To obtain the DAGs, we used extensive expert elicitation (exploiting Australia's exceptionally low COVID-19 burden) in structured online sessions. Groups of clinical and other specialists were enlisted to filter, interpret and discuss the literature and develop a current consensus. We encouraged inclusion of theoretically salient latent (unobservable) variables, likely mechanisms by extrapolation from other diseases, and documented supporting literature while noting controversies. Our method was iterative and incremental: systematically refining and validating the group output using one-on-one follow-up meetings with original and new experts. 35 experts contributed 126 hours face-to-face, and could review our products. RESULTS: We present two key models, for the initial infection of the respiratory tract and the possible progression to complications, as causal DAGs and BNs with corresponding verbal descriptions, dictionaries and sources. These are the first published causal models of COVID-19 pathophysiology. CONCLUSIONS: Our method demonstrates an improved procedure for developing BNs via expert elicitation, which other teams can implement to model emergent complex phenomena. Our results have three anticipated applications: (i) freely disseminating updatable expert knowledge; (ii) guiding design and analysis of observational and clinical studies; (iii) developing and validating automated tools for causal reasoning and decision support. We are developing such tools for the initial diagnosis, resource management, and prognosis of COVID-19, parameterized using the ISARIC and LEOSS databases.

BARD: A Structured Technique for Group Elicitation of Bayesian Networks to Support Analytic Reasoning.

In many complex, real-world situations, problem solving and decision making require effective reasoning about causation and uncertainty. However, human reasoning in these cases is prone to confusion and error. Bayesian networks (BNs) are an artificial intelligence technology that models uncertain situations, supporting better probabilistic and causal reasoning and decision making. However, to date, BN methodologies and software require (but do not include) substantial upfront training, do not provide much guidance on either the model building process or on using the model for reasoning and reporting, and provide no support for building BNs collaboratively. Here, we contribute a detailed description and motivation for our new methodology and application, Bayesian ARgumentation via Delphi (BARD). BARD utilizes BNs and addresses these shortcomings by integrating (1) short, high-quality e-courses, tips, and help on demand; (2) a stepwise, iterative, and incremental BN construction process; (3) report templates and an automated explanation tool; and (4) a multiuser web-based software platform and Delphi-style social processes. The result is an end-to-end online platform, with associated online training, for groups without prior BN expertise to understand and analyze a problem, build a model of its underlying probabilistic causal structure, validate and reason with the causal model, and (optionally) use it to produce a written analytic report. Initial experiments demonstrate that, for suitable problems, BARD aids in reasoning and reporting. Comparing their effect sizes also suggests BARD's BN-building and collaboration combine beneficially and cumulatively.

Predicting reliability through structured expert elicitation with the repliCATS (Collaborative Assessments for Trustworthy Science) process.

As replications of individual studies are resource intensive, techniques for predicting the replicability are required. We introduce the repliCATS (Collaborative Assessments for Trustworthy Science) process, a new method for eliciting expert predictions about the replicability of research. This process is a structured expert elicitation approach based on a modified Delphi technique applied to the evaluation of research claims in social and behavioural sciences. The utility of processes to predict replicability is their capacity to test scientific claims without the costs of full replication. Experimental data supports the validity of this process, with a validation study producing a classification accuracy of 84% and an Area Under the Curve of 0.94, meeting or exceeding the accuracy of other techniques used to predict replicability. The repliCATS process provides other benefits. It is highly scalable, able to be deployed for both rapid assessment of small numbers of claims, and assessment of high volumes of claims over an extended period through an online elicitation platform, having been used to assess 3000 research claims over an 18 month period. It is available to be implemented in a range of ways and we describe one such implementation. An important advantage of the repliCATS process is that it collects qualitative data that has the potential to provide insight in understanding the limits of generalizability of scientific claims. The primary limitation of the repliCATS process is its reliance on human-derived predictions with consequent costs in terms of participant fatigue although careful design can minimise these costs. The repliCATS process has potential applications in alternative peer review and in the allocation of effort for replication studies.

Individuals vs. BARD: Experimental Evaluation of an Online System for Structured, Collaborative Bayesian Reasoning.

US intelligence analysts must weigh up relevant evidence to assess the probability of their conclusions, and express this reasoning clearly in written reports for decision-makers. Typically, they work alone with no special analytic tools, and sometimes succumb to common probabilistic and causal reasoning errors. So, the US government funded a major research program (CREATE) for four large academic teams to develop new structured, collaborative, software-based methods that might achieve better results. Our team's method (BARD) is the first to combine two key techniques: constructing causal Bayesian network models (BNs) to represent analyst knowledge, and small-group collaboration via the Delphi technique. BARD also incorporates compressed, high-quality online training allowing novices to use it, and checklist-inspired report templates with a rudimentary AI tool for generating text explanations from analysts' BNs. In two prior experiments, our team showed BARD's BN-building assists probabilistic reasoning when used by individuals, with a large effect (Glass' Δ 0.8) (Cruz et al., 2020), and even minimal Delphi-style interactions improve the BN structures individuals produce, with medium to very large effects (Glass' Δ 0.5-1.3) (Bolger et al., 2020). This experiment is the critical test of BARD as an integrated system and possible alternative to business-as-usual for intelligence analysis. Participants were asked to solve three probabilistic reasoning problems spread over 5 weeks, developed by our team to test both quantitative accuracy and susceptibility to tempting qualitative fallacies. Our 256 participants were randomly assigned to form 25 teams of 6-9 using BARD and 58 individuals using Google Suite and (if desired) the best pen-and-paper techniques. For each problem, BARD outperformed this control with very large to huge effects (Glass' Δ 1.4-2.2), greatly exceeding CREATE's initial target. We conclude that, for suitable problems, BARD already offers significant advantages over both business-as-usual and existing BN software. Our effect sizes also suggest BARD's BN-building and collaboration combined beneficially and cumulatively, although implementation differences decreased performances compared to Cruz et al. (2020), so interaction may have contributed. BARD has enormous potential for further development and testing of specific components and on more complex problems, and many potential applications beyond intelligence analysis.

Nitrofurazone quantification in milk at the European Union minimum required performance limit of 1 ng g(-1): circumventing the semicarbazide problem.

Nitrofurazone is an antibiotic with carcinogenic properties. Efforts by regulatory authorities to control nitrofurazone from agricultural foods are an important public health measure that have, to some extent, been undermined by widespread use amongst laboratories of the unreliable marker metabolite semicarbazide. This work confirms what has long been suspected, namely that powdered dairy products that are initially free of semicarbazide develop semicarbazide under storage conditions such as occur normally across commercial supply chains. The low ng g(-)(1) levels of semicarbazide formed in this way are insufficient to present any food safety hazard. That such development of a marker metabolite is demonstrated to occur by innocent means effectively invalidates the use of semicarbazide as a marker metabolite for powdered dairy products, and exacerbates the regulatory need for a more suitable analytical methodology. In milk, unlike meat, nitrofurazone is known to remain stable and thus available for analysis in the intact form, rather than necessitating any use of a metabolite or fragment. However, no previous methodology that was capable of achieving the stringent European minimum required performance limit of 1 ng g(-)(1) when using intact nitrofurazone had been described for milk. This work describes a specific methodology using LC-MS/MS for milk and milk powder; it achieves detection of intact nitrofurazone (as well as furazolidone, furaltadone and nitrofurantoin) to levels well below 1 ng g(-)(1). Laboratories will no longer need to use semicarbazide as an unreliable marker metabolite for the analysis of nitrofurazone in dairy products, paving the way for regulatory authorities to better control nitrofurazone abuse with greater confidence.

Effect of affordable technology on physical activity levels and mobility outcomes in rehabilitation: a protocol for the Activity and MObility UsiNg Technology (AMOUNT) rehabilitation trial.

INTRODUCTION: People with mobility limitations can benefit from rehabilitation programmes that provide a high dose of exercise. However, since providing a high dose of exercise is logistically challenging and resource-intensive, people in rehabilitation spend most of the day inactive. This trial aims to evaluate the effect of the addition of affordable technology to usual care on physical activity and mobility in people with mobility limitations admitted to inpatient aged and neurological rehabilitation units compared to usual care alone. METHODS AND ANALYSIS: A pragmatic, assessor blinded, parallel-group randomised trial recruiting 300 consenting rehabilitation patients with reduced mobility will be conducted. Participants will be individually randomised to intervention or control groups. The intervention group will receive technology-based exercise to target mobility and physical activity problems for 6 months. The technology will include the use of video and computer games/exercises and tablet applications as well as activity monitors. The control group will not receive any additional intervention and both groups will receive usual inpatient and outpatient rehabilitation care over the 6-month study period. The coprimary outcomes will be objectively assessed physical activity (proportion of the day spent upright) and mobility (Short Physical Performance Battery) at 6 months after randomisation. Secondary outcomes will include: self-reported and objectively assessed physical activity, mobility, cognition, activity performance and participation, utility-based quality of life, balance confidence, technology self-efficacy, falls and service utilisation. Linear models will assess the effect of group allocation for each continuously scored outcome measure with baseline scores entered as a covariate. Fall rates between groups will be compared using negative binomial regression. Primary analyses will be preplanned, conducted while masked to group allocation and use an intention-to-treat approach. ETHICS AND DISSEMINATION: The protocol has been approved by the relevant Human Research Ethics Committees and the results will be disseminated widely through peer-reviewed publication and conference presentations. TRIAL REGISTRATION NUMBER: ACTRN12614000936628. Pre-results.