BACKGROUND: Identifying and managing serious spinal pathology (SSP) such as cauda equina syndrome or spinal infection in patients presenting with low back pain is challenging. Traditional red flag questioning is increasingly criticized, and previous studies show that many clinicians lack confidence in managing patients presenting with red flags. Improving decision-making and reducing the variability of care for these patients is a key priority for clinicians and researchers. OBJECTIVE: We aimed to improve SSP identification by constructing and validating a decision support tool using a Bayesian network (BN), which is an artificial intelligence technique that combines current evidence and expert knowledge. METHODS: A modified RAND appropriateness procedure was undertaken with 16 experts over 3 rounds, designed to elicit the variables, structure, and conditional probabilities necessary to build a causal BN. The BN predicts the likelihood of a patient with a particular presentation having an SSP. The second part of this study used an established framework to direct a 4-part validation that included comparison of the BN with consensus statements, practice guidelines, and recent research. Clinical cases were entered into the model and the results were compared with clinical judgment from spinal experts who were not involved in the elicitation. Receiver operating characteristic curves were plotted and area under the curve were calculated for accuracy statistics. RESULTS: The RAND appropriateness procedure elicited a model including 38 variables in 3 domains: risk factors (10 variables), signs and symptoms (17 variables), and judgment factors (11 variables). Clear consensus was found in the risk factors and signs and symptoms for SSP conditions. The 4-part BN validation demonstrated good performance overall and identified areas for further development. Comparison with available clinical literature showed good overall agreement but suggested certain improvements required to, for example, 2 of the 11 judgment factors. Case analysis showed that cauda equina syndrome, space-occupying lesion/cancer, and inflammatory condition identification performed well across the validation domains. Fracture identification performed less well, but the reasons for the erroneous results are well understood. A review of the content by independent spinal experts backed up the issues with the fracture node, but the BN was otherwise deemed acceptable. CONCLUSIONS: The RAND appropriateness procedure and validation framework were successfully implemented to develop the BN for SSP. In comparison with other expert-elicited BN studies, this work goes a step further in validating the output before attempting implementation. Using a framework for model validation, the BN showed encouraging validity and has provided avenues for further developing the outputs that demonstrated poor accuracy. This study provides the vital first step of improving our ability to predict outcomes in low back pain by first considering the problem of SSP. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): RR2-10.2196/21804.
Patient Reported Outcome Measures (PROMs) are questionnaires completed by patients about aspects of their health status. They are a vital part of learning health systems as they are the primary source of information about important outcomes that are best assessed by patients such as pain, disability, anxiety and depression. The volume of questions can easily become burdensome. Previous techniques reduced this burden by dynamically selecting questions from question item banks which are specifically built for different latent constructs being measured. These techniques analyzed the information function between each question in the item bank and the measured construct based on item response theory then used this information function to dynamically select questions by computerized adaptive testing. Here we extend those ideas by using Bayesian Networks (BNs) to enable Computerized Adaptive Testing (CAT) for efficient and accurate question selection on widely-used existing PROMs. BNs offer more comprehensive probabilistic models of the connections between different PROM questions, allowing the use of information theoretic techniques to select the most informative questions. We tested our methods using five clinical PROM datasets, demonstrating that answering a small subset of questions selected with CAT has similar predictions and error to answering all questions in the PROM BN. Our results show that answering 30% - 75% questions selected with CAT had an average area under the receiver operating characteristic curve (AUC) of 0.92 (min: 0.8 - max: 0.98) for predicting the measured constructs. BNs outperformed alternative CAT approaches with a 5% (min: 0.01% - max: 9%) average increase in the accuracy of predicting the responses to unanswered question items.
Health Status , Patient Reported Outcome Measures , Bayes Theorem , Reproducibility of Results , Surveys and Questionnaires
BACKGROUND: Low back pain (LBP) is an increasingly burdensome condition for patients and health professionals alike, with consistent demonstration of increasing persistent pain and disability. Previous decision support tools for LBP management have focused on a subset of factors owing to time constraints and ease of use for the clinician. With the explosion of interest in machine learning tools and the commitment from Western governments to introduce this technology, there are opportunities to develop intelligent decision support tools. We will do this for LBP using a Bayesian network, which will entail constructing a clinical reasoning model elicited from experts. OBJECTIVE: This paper proposes a method for conducting a modified RAND appropriateness procedure to elicit the knowledge required to construct a Bayesian network from a group of domain experts in LBP, and reports the lessons learned from the internal pilot of the procedure. METHODS: We propose to recruit expert clinicians with a special interest in LBP from across a range of medical specialties, such as orthopedics, rheumatology, and sports medicine. The procedure will consist of four stages. Stage 1 is an online elicitation of variables to be considered by the model, followed by a face-to-face workshop. Stage 2 is an online elicitation of the structure of the model, followed by a face-to-face workshop. Stage 3 consists of an online phase to elicit probabilities to populate the Bayesian network. Stage 4 is a rudimentary validation of the Bayesian network. RESULTS: Ethical approval has been obtained from the Research Ethics Committee at Queen Mary University of London. An internal pilot of the procedure has been run with clinical colleagues from the research team. This showed that an alternating process of three remote activities and two in-person meetings was required to complete the elicitation without overburdening participants. Lessons learned have included the need for a bespoke online elicitation tool to run between face-to-face meetings and for careful operational definition of descriptive terms, even if widely clinically used. Further, tools are required to remotely deliver training about self-identification of various forms of cognitive bias and explain the underlying principles of a Bayesian network. The use of the internal pilot was recognized as being a methodological necessity. CONCLUSIONS: We have proposed a method to construct Bayesian networks that are representative of expert clinical reasoning for a musculoskeletal condition in this case. We have tested the method with an internal pilot to refine the process prior to deployment, which indicates the process can be successful. The internal pilot has also revealed the software support requirements for the elicitation process to model clinical reasoning for a range of conditions. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): DERR1-10.2196/21804.
