From pre-test and post-test probabilities to medical decision making.

BACKGROUND: A central goal of modern evidence-based medicine is the development of simple and easy to use tools that help clinicians integrate quantitative information into medical decision-making. The Bayesian Pre-test/Post-test Probability (BPP) framework is arguably the most well known of such tools and provides a formal approach to quantify diagnostic uncertainty given the result of a medical test or the presence of a clinical sign. Yet, clinical decision-making goes beyond quantifying diagnostic uncertainty and requires that that uncertainty be balanced against the various costs and benefits associated with each possible decision. Despite increasing attention in recent years, simple and flexible approaches to quantitative clinical decision-making have remained elusive. METHODS: We extend the BPP framework using concepts of Bayesian Decision Theory. By integrating cost, we can expand the BPP framework to allow for clinical decision-making. RESULTS: We develop a simple quantitative framework for binary clinical decisions (e.g., action/inaction, treat/no-treat, test/no-test). Let p be the pre-test or post-test probability that a patient has disease. We show that r ∗ = ( 1 - p ) / p represents a critical value called a decision boundary. In terms of the relative cost of under- to over-acting, r ∗ represents the critical value at which action and inaction are equally optimal. We demonstrate how this decision boundary can be used at the bedside through case studies and as a research tool through a reanalysis of a recent study which found widespread misestimation of pre-test and post-test probabilities among clinicians. CONCLUSIONS: Our approach is so simple that it should be thought of as a core, yet previously overlooked, part of the BPP framework. Unlike prior approaches to quantitative clinical decision-making, our approach requires little more than a hand-held calculator, is applicable in almost any setting where the BPP framework can be used, and excels in situations where the costs and benefits associated with a particular decision are patient-specific and difficult to quantify.

Neural correlates of valence-dependent belief and value updating during uncertainty reduction: An fNIRS study.

Selective use of new information is crucial for adaptive decision-making. Combining a gamble bidding task with assessing cortical responses using functional near-infrared spectroscopy (fNIRS), we investigated potential effects of information valence on behavioral and neural processes of belief and value updating during uncertainty reduction in young adults. By modeling changes in the participants' expressed subjective values using a Bayesian model, we dissociated processes of (i) updating beliefs about statistical properties of the gamble, (ii) updating values of a gamble based on new information about its winning probabilities, as well as (iii) expectancy violation. The results showed that participants used new information to update their beliefs and values about the gambles in a quasi-optimal manner, as reflected in the selective updating only in situations with reducible uncertainty. Furthermore, their updating was valence-dependent: information indicating an increase in winning probability was underweighted, whereas information about a decrease in winning probability was updated in good agreement with predictions of the Bayesian decision theory. Results of model-based and moderation analyses showed that this valence-dependent asymmetry was associated with a distinct contribution of expectancy violation, besides belief updating, to value updating after experiencing new positive information regarding winning probabilities. In line with the behavioral results, we replicated previous findings showing involvements of frontoparietal brain regions in the different components of updating. Furthermore, this study provided novel results suggesting a valence-dependent recruitment of brain regions. Individuals with stronger oxyhemoglobin responses during value updating was more in line with predictions of the Bayesian model while integrating new information that indicates an increase in winning probability. Taken together, this study provides first results showing expectancy violation as a contributing factor to sub-optimal valence-dependent updating during uncertainty reduction and suggests limitations of normative Bayesian decision theory.

A decision-theoretic approach to Bayesian clinical trial design and evaluation of robustness to prior-data conflict.

Bayesian clinical trials allow taking advantage of relevant external information through the elicitation of prior distributions, which influence Bayesian posterior parameter estimates and test decisions. However, incorporation of historical information can have harmful consequences on the trial's frequentist (conditional) operating characteristics in case of inconsistency between prior information and the newly collected data. A compromise between meaningful incorporation of historical information and strict control of frequentist error rates is therefore often sought. Our aim is thus to review and investigate the rationale and consequences of different approaches to relaxing strict frequentist control of error rates from a Bayesian decision-theoretic viewpoint. In particular, we define an integrated risk which incorporates losses arising from testing, estimation, and sampling. A weighted combination of the integrated risk addends arising from testing and estimation allows moving smoothly between these two targets. Furthermore, we explore different possible elicitations of the test error costs, leading to test decisions based either on posterior probabilities, or solely on Bayes factors. Sensitivity analyses are performed following the convention which makes a distinction between the prior of the data-generating process, and the analysis prior adopted to fit the data. Simulation in the case of normal and binomial outcomes and an application to a one-arm proof-of-concept trial, exemplify how such analysis can be conducted to explore sensitivity of the integrated risk, the operating characteristics, and the optimal sample size, to prior-data conflict. Robust analysis prior specifications, which gradually discount potentially conflicting prior information, are also included for comparison. Guidance with respect to cost elicitation, particularly in the context of a Phase II proof-of-concept trial, is provided.

Hospital profiling using Bayesian decision theory.

For evaluating the quality of care provided by hospitals, special interest lies in the identification of performance outliers. The classification of healthcare providers as outliers or non-outliers is a decision under uncertainty, because the true quality is unknown and can only be inferred from an observed result of a quality indicator. We propose to embed the classification of healthcare providers into a Bayesian decision theoretical framework that enables the derivation of optimal decision rules with respect to the expected decision consequences. We propose paradigmatic utility functions for two typical purposes of hospital profiling: the external reporting of healthcare quality and the initiation of change in care delivery. We make use of funnel plots to illustrate and compare the resulting optimal decision rules and argue that sensitivity and specificity of the resulting decision rules should be analyzed. We then apply the proposed methodology to the area of hip replacement surgeries by analyzing data from 1,277 hospitals in Germany which performed over 180,000 such procedures in 2017. Our setting illustrates that the classification of outliers can be highly dependent upon the underlying utilities. We conclude that analyzing the classification of hospitals as a decision theoretic problem helps to derive transparent and justifiable decision rules. The methodology for classifying quality indicator results is implemented in an R package (iqtigbdt) and is available on GitHub.

I want to believe: delusion, motivated reasoning, and Bayesian decision theory.

Introduction: Several arguments suggest that motivated reasoning (occurring when beliefs are not solely shaped by accuracy, but also by other motives such as promoting self-esteem or self-protection) is important in delusions. However, classical theories of delusion disregard the role of motivated reasoning. Thus, this role remains poorly understood.Methods: To explore the role of motivated reasoning in delusions, here we propose a computational model of delusion based on a Bayesian decision framework. This proposes that beliefs are not only evaluated based on their accuracy (as in classical theories), but also based on the cost (in terms of reward and punishment) of rejecting them.Results: The model proposes that, when the values at stake are high (as often it is the case in the context of delusion), a belief might be endorsed because rejecting it is evaluated as too costly, even if the belief is less accurate. This process might contribute to the genesis of delusions.Conclusions: Our account offers an interpretation of how motivated reasoning might shape delusions. This can inspire research on the affective and motivational processes supporting delusions in clinical conditions such as in psychosis, neurological disorders, and delusional disorder.

Fitness Beats Truth in the Evolution of Perception.

Does natural selection favor veridical percepts-those that accurately (if not exhaustively) depict objective reality? Perceptual and cognitive scientists standardly claim that it does. Here we formalize this claim using the tools of evolutionary game theory and Bayesian decision theory. We state and prove the "Fitness-Beats-Truth (FBT) Theorem" which shows that the claim is false: If one starts with the assumption that perception involves inference to states of the objective world, then the FBT Theorem shows that a strategy that simply seeks to maximize expected-fitness payoff, with no attempt to estimate the "true" world state, does consistently better. More precisely, the FBT Theorem provides a quantitative measure of the extent to which the fitness-only strategy dominates the truth strategy, and of how this dominance increases with the size of the perceptual space. The FBT Theorem supports the Interface Theory of Perception (e.g. Hoffman et al. in Psychon Bull Rev https://doi.org/10.3758/s13423-015-0890-8 , 2015), which proposes that our perceptual systems have evolved to provide a species-specific interface to guide adaptive behavior, and not to provide a veridical representation of objective reality.

Fact, Fiction, and Fitness.

A theory of consciousness, whatever else it may do, must address the structure of experience. Our perceptual experiences are richly structured. Simply seeing a red apple, swaying between green leaves on a stout tree, involves symmetries, geometries, orders, topologies, and algebras of events. Are these structures also present in the world, fully independent of their observation? Perceptual theorists of many persuasions-from computational to radical embodied-say yes: perception veridically presents to observers structures that exist in an observer-independent world; and it does so because natural selection shapes perceptual systems to be increasingly veridical. Here we study four structures: total orders, permutation groups, cyclic groups, and measurable spaces. We ask whether the payoff functions that drive evolution by natural selection are homomorphisms of these structures. We prove, in each case, that generically the answer is no: as the number of world states and payoff values go to infinity, the probability that a payoff function is a homomorphism goes to zero. We conclude that natural selection almost surely shapes perceptions of these structures to be non-veridical. This is consistent with the interface theory of perception, which claims that natural selection shapes perceptual systems not to provide veridical perceptions, but to serve as species-specific interfaces that guide adaptive behavior. Our results present a constraint for any theory of consciousness which assumes that structure in perceptual experience is shaped by natural selection.

Determining the optimal forensic DNA analysis procedure following investigation of sample quality.

Crime scene traces of various types are routinely sent to forensic laboratories for analysis, generally with the aim of addressing questions about the source of the trace. The laboratory may choose to analyse the samples in different ways depending on the type and quality of the sample, the importance of the case and the cost and performance of the available analysis methods. Theoretically well-founded guidelines for the choice of analysis method are, however, lacking in most situations. In this paper, it is shown how such guidelines can be created using Bayesian decision theory. The theory is applied to forensic DNA analysis, showing how the information from the initial qPCR analysis can be utilized. It is assumed the alternatives for analysis are using a standard short tandem repeat (STR) DNA analysis assay, using the standard assay and a complementary assay, or the analysis may be cancelled following quantification. The decision is based on information about the DNA amount and level of DNA degradation of the forensic sample, as well as case circumstances and the cost for analysis. Semi-continuous electropherogram models are used for simulation of DNA profiles and for computation of likelihood ratios. It is shown how tables and graphs, prepared beforehand, can be used to quickly find the optimal decision in forensic casework.

Decision-theoretic designs for a series of trials with correlated treatment effects using the Sarmanov multivariate beta-binomial distribution.

The motivation for the work in this article is the setting in which a number of treatments are available for evaluation in phase II clinical trials and where it may be infeasible to try them concurrently because the intended population is small. This paper introduces an extension of previous work on decision-theoretic designs for a series of phase II trials. The program encompasses a series of sequential phase II trials with interim decision making and a single two-arm phase III trial. The design is based on a hybrid approach where the final analysis of the phase III data is based on a classical frequentist hypothesis test, whereas the trials are designed using a Bayesian decision-theoretic approach in which the unknown treatment effect is assumed to follow a known prior distribution. In addition, as treatments are intended for the same population it is not unrealistic to consider treatment effects to be correlated. Thus, the prior distribution will reflect this. Data from a randomized trial of severe arthritis of the hip are used to test the application of the design. We show that the design on average requires fewer patients in phase II than when the correlation is ignored. Correspondingly, the time required to recommend an efficacious treatment for phase III is quicker.

Suboptimality in perceptual decision making.

Human perceptual decisions are often described as optimal. Critics of this view have argued that claims of optimality are overly flexible and lack explanatory power. Meanwhile, advocates for optimality have countered that such criticisms single out a few selected papers. To elucidate the issue of optimality in perceptual decision making, we review the extensive literature on suboptimal performance in perceptual tasks. We discuss eight different classes of suboptimal perceptual decisions, including improper placement, maintenance, and adjustment of perceptual criteria; inadequate tradeoff between speed and accuracy; inappropriate confidence ratings; misweightings in cue combination; and findings related to various perceptual illusions and biases. In addition, we discuss conceptual shortcomings of a focus on optimality, such as definitional difficulties and the limited value of optimality claims in and of themselves. We therefore advocate that the field drop its emphasis on whether observed behavior is optimal and instead concentrate on building and testing detailed observer models that explain behavior across a wide range of tasks. To facilitate this transition, we compile the proposed hypotheses regarding the origins of suboptimal perceptual decisions reviewed here. We argue that verifying, rejecting, and expanding these explanations for suboptimal behavior - rather than assessing optimality per se - should be among the major goals of the science of perceptual decision making.

Analysing and exemplifying forensic conclusion criteria in terms of Bayesian decision theory.

There is ongoing discussion in forensic science and the law about the nature of the conclusions reached based on scientific evidence, and on how such conclusions - and conclusion criteria - may be justified by rational argument. Examples, among others, are encountered in fields such as fingermarks (e.g., 'this fingermark comes from Mr. A's left thumb'), handwriting examinations (e.g., 'the questioned signature is that of Mr. A'), kinship analyses (e.g., 'Mr. A is the father of child C') or anthropology (e.g., 'these are human remains'). Considerable developments using formal methods of reasoning based on, for example (Bayesian) decision theory, are available in literature, but currently such reference principles are not explicitly used in operational forensic reporting and ensuing decision-making. Moreover, applied examples, illustrating the principles, are scarce. A potential consequence of this in practical proceedings, and hence a cause of concern, is that underlying ingredients of decision criteria (such as losses quantifying the undesirability of adverse decision consequences), are not properly dealt with. There is merit, thus, in pursuing the study and discussion of practical examples, demonstrating that formal decision-theoretic principles are not merely conceptual considerations. Actually, these principles can be shown to underpin practical decision-making procedures and existing legal decision criteria, though often not explicitly apparent as such. In this paper, we will present such examples and discuss their properties from a Bayesian decision-theoretic perspective. We will argue that these are essential concepts for an informed discourse on decision-making across forensic disciplines and the development of a coherent view on this topic. We will also emphasize that these principles are of normative nature in the sense that they provide standards against which actual judgment and decision-making may be compared. Most importantly, these standards are justified independently of peoples' observable decision behaviour, and of whether or not one endorses these formal methods of reasoning.

Likelihood Ratio as Weight of Forensic Evidence: A Closer Look.

The forensic science community has increasingly sought quantitative methods for conveying the weight of evidence. Experts from many forensic laboratories summarize their findings in terms of a likelihood ratio. Several proponents of this approach have argued that Bayesian reasoning proves it to be normative. We find this likelihood ratio paradigm to be unsupported by arguments of Bayesian decision theory, which applies only to personal decision making and not to the transfer of information from an expert to a separate decision maker. We further argue that decision theory does not exempt the presentation of a likelihood ratio from uncertainty characterization, which is required to assess the fitness for purpose of any transferred quantity. We propose the concept of a lattice of assumptions leading to an uncertainty pyramid as a framework for assessing the uncertainty in an evaluation of a likelihood ratio. We demonstrate the use of these concepts with illustrative examples regarding the refractive index of glass and automated comparison scores for fingerprints.

Neural mechanisms for integrating prior knowledge and likelihood in value-based probabilistic inference.

In Bayesian decision theory, knowledge about the probabilities of possible outcomes is captured by a prior distribution and a likelihood function. The prior reflects past knowledge and the likelihood summarizes current sensory information. The two combined (integrated) form a posterior distribution that allows estimation of the probability of different possible outcomes. In this study, we investigated the neural mechanisms underlying Bayesian integration using a novel lottery decision task in which both prior knowledge and likelihood information about reward probability were systematically manipulated on a trial-by-trial basis. Consistent with Bayesian integration, as sample size increased, subjects tended to weigh likelihood information more compared with prior information. Using fMRI in humans, we found that the medial prefrontal cortex (mPFC) correlated with the mean of the posterior distribution, a statistic that reflects the integration of prior knowledge and likelihood of reward probability. Subsequent analysis revealed that both prior and likelihood information were represented in mPFC and that the neural representations of prior and likelihood in mPFC reflected changes in the behaviorally estimated weights assigned to these different sources of information in response to changes in the environment. Together, these results establish the role of mPFC in prior-likelihood integration and highlight its involvement in representing and integrating these distinct sources of information.

The cost of misremembering: Inferring the loss function in visual working memory.

Visual working memory (VWM) is a highly limited storage system. A basic consequence of this fact is that visual memories cannot perfectly encode or represent the veridical structure of the world. However, in natural tasks, some memory errors might be more costly than others. This raises the intriguing possibility that the nature of memory error reflects the costs of committing different kinds of errors. Many existing theories assume that visual memories are noise-corrupted versions of afferent perceptual signals. However, this additive noise assumption oversimplifies the problem. Implicit in the behavioral phenomena of visual working memory is the concept of a loss function: a mathematical entity that describes the relative cost to the organism of making different types of memory errors. An optimally efficient memory system is one that minimizes the expected loss according to a particular loss function, while subject to a constraint on memory capacity. This paper describes a novel theoretical framework for characterizing visual working memory in terms of its implicit loss function. Using inverse decision theory, the empirical loss function is estimated from the results of a standard delayed recall visual memory experiment. These results are compared to the predicted behavior of a visual working memory system that is optimally efficient for a previously identified natural task, gaze correction following saccadic error. Finally, the approach is compared to alternative models of visual working memory, and shown to offer a superior account of the empirical data across a range of experimental datasets.

From Pre-test and Post-test Probabilities to Medical Decision Making.

Background: A central goal of modern evidence-based medicine is the development of simple and easy to use tools that help clinicians integrate quantitative information into medical decision-making. The Bayesian Pre-test/Post-test Probability (BPP) framework is arguably the most well known of such tools and provides a formal approach to quantify diagnostic uncertainty given the result of a medical test or the presence of a clinical sign. Yet, clinical decision-making goes beyond quantifying diagnostic uncertainty and requires that that uncertainty be balanced against the various costs and benefits associated with each possible decision. Despite increasing attention in recent years, simple and flexible approaches to quantitative clinical decision-making have remained elusive. Methods: We extend the BPP framework using concepts of Bayesian Decision Theory. By integrating cost, we can expand the BPP framework to allow for clinical decision-making. Results: We develop a simple quantitative framework for binary clinical decisions (e.g., action/inaction, treat/no-treat, test/no-test). Let p be the pre-test or post-test probability that a patient has disease. We show that r*=(1-p)/p represents a critical value called a decision boundary. In terms of the relative cost of under- to over-acting, r* represents the critical value at which action and inaction are equally optimal. We demonstrate how this decision boundary can be used at the bedside through case studies and as a research tool through a reanalysis of a recent study which found widespread misestimation of pre-test and post-test probabilities among clinicians. Conclusions: Our approach is so simple that it should be thought of as a core, yet previously overlooked, part of the BPP framework. Unlike prior approaches to quantitative clinical decision-making, our approach requires little more than a hand-held calculator, is applicable in almost any setting where the BPP framework can be used, and excels in situations where the costs and benefits associated with a particular decision are patient-specific and difficult to quantify.

Subtleties in Bayesian decision-theoretic analysis for forensic findings: Notes on recent discussion of the role of validation study data in rational decision making.

This technical note extends a recent discussion in this journal of the role of validation study data in rational decision making. One argument that has been made in this context, using elements of Bayesian decision theory, is that further aggregation of validation study data into error rates involves a loss of information that compromises rational inference and decision making and should therefore be discouraged. This technical note seeks to explain that this argument can be developed at different levels of detail, depending on the definition of the propositions of interest, the forensic findings to be evaluated (and hence the form of the likelihood ratio), and the characterization of the relative desirability of decision consequences. The analyses proposed here reveal the cascade of abstractions and assumptions into which discussions about the use of validation study results in forensic science have fallen. This reinforces the conclusion that further aggregation of validation study data into error rates is problematic. It also suggests that even if a definition of error rate(s) could be agreed upon and defensively quantified in a given application, we should rethink and possibly adjust our expectations about what exactly error rates can practically contribute to rational modes of reasoning and decision making in legal contexts.

Optimizing Genomic Parental Selection for Categorical and Continuous-Categorical Multi-Trait Mixtures.

This study presents a novel approach for the optimization of genomic parental selection in breeding programs involving categorical and continuous-categorical multi-trait mixtures (CMs and CCMMs). Utilizing the Bayesian decision theory (BDT) and latent trait models within a multivariate normal distribution framework, we address the complexities of selecting new parental lines across ordinal and continuous traits for breeding. Our methodology enhances precision and flexibility in genetic selection, validated through extensive simulations. This unified approach presents significant potential for the advancement of genetic improvements in diverse breeding contexts, underscoring the importance of integrating both categorical and continuous traits in genomic selection frameworks.

Bayesian analysis and classification of two enzyme-linked immunosorbent assay tests without a gold standard.

Reconciling two quantitative enzyme-linked immunosorbent assay tests for an antibody to an RNA virus, in a situation without a gold standard and where false negatives may occur, is the motivation for this work. False negatives occur when access of the antibody to the binding site is blocked. On the basis of the mechanism of the assay, a mixture of four bivariate normal distributions is proposed with the mixture probabilities depending on a two-stage latent variable model including the prevalence of the antibody in the population and the probabilities of blocking on each test. There is prior information on the prevalence of the antibody, and also on the probability of false negatives, and so a Bayesian analysis is used. The dependence between the two tests is modeled to be consistent with the biological mechanism. Bayesian decision theory is utilized for classification.The proposed method is applied to the motivating data set to classify the data into two groups: those with and those without the antibody. Simulation studies describe the properties of the estimation and the classification. Sensitivity to the choice of the prior distribution is also addressed by simulation. The same model with two levels of latent variables is applicable in other testing procedures such as quantitative polymerase chain reaction tests, where false negatives occur when there is a mutation in the primer sequence.

Estimating Value-Based Price and Quantifying Uncertainty around It in Health Technology Assessment: Frequentist and Bayesian Approaches.

BACKGROUND: Although several statistical methods have been developed to inform decision making on reimbursement under uncertainty (e.g., expected net benefit, cost-effectiveness acceptability curves, and expected value of perfect information [EVPI]), those for value-based pricing are limited. This research develops methods for estimating the value-based price and quantifying the uncertainty around it in health technology assessment. METHODS: We defined the value-based price of a medical product under assessment as the price at which the incremental cost-effectiveness ratio is just equal to a cost-effectiveness threshold. According to this definition, we derived an explicit form of the value-based price. Using this explicit form, we developed frequentist and Bayesian approaches to value-based pricing under uncertainty. Our proposed methods were illustrated via 2 hypothetical case studies. RESULTS: The value-based price can be expressed explicitly using cost, effectiveness, and a cost-effectiveness threshold and is a linear function of a cost-effectiveness threshold. In the frequentist framework, point estimation, interval estimation, and hypothesis testing for the value-based price are available. In the Bayesian framework, the best estimate of the value-based price under uncertainty is the weighted median value-based price with the weight of the expected consumption volume of a medical product under assessment. This is based on the opportunity loss incurred by a decision error in value-based pricing. This opportunity loss also provides a basis for the calculation of EVPI associated with value-based pricing. These methods provided estimates of the value-based prices of medical products and the uncertainty around them in 2 hypothetical case studies. CONCLUSIONS: Our developed methods can improve decision making on value-based pricing in health technology assessment.

Bayesian decision theory and navigation.

Spatial navigation is a complex cognitive activity that depends on perception, action, memory, reasoning, and problem-solving. Effective navigation depends on the ability to combine information from multiple spatial cues to estimate one's position and the locations of goals. Spatial cues include landmarks, and other visible features of the environment, and body-based cues generated by self-motion (vestibular, proprioceptive, and efferent information). A number of projects have investigated the extent to which visual cues and body-based cues are combined optimally according to statistical principles. Possible limitations of these investigations are that they have not accounted for navigators' prior experiences with or assumptions about the task environment and have not tested complete decision models. We examine cue combination in spatial navigation from a Bayesian perspective and present the fundamental principles of Bayesian decision theory. We show that a complete Bayesian decision model with an explicit loss function can explain a discrepancy between optimal cue weights and empirical cues weights observed by (Chen et al. Cognitive Psychology, 95, 105-144, 2017) and that the use of informative priors to represent cue bias can explain the incongruity between heading variability and heading direction observed by (Zhao and Warren 2015b, Psychological Science, 26[6], 915-924). We also discuss (Petzschner and Glasauer's , Journal of Neuroscience, 31(47), 17220-17229, 2011) use of priors to explain biases in estimates of linear displacements during visual path integration. We conclude that Bayesian decision theory offers a productive theoretical framework for investigating human spatial navigation and believe that it will lead to a deeper understanding of navigational behaviors.