Advancing unanchored simulated treatment comparisons: A novel implementation and simulation study.

Population-adjusted indirect comparisons, developed in the 2010s, enable comparisons between two treatments in different studies by balancing patient characteristics in the case where individual patient-level data (IPD) are available for only one study. Health technology assessment (HTA) bodies increasingly rely on these methods to inform funding decisions, typically using unanchored indirect comparisons (i.e., without a common comparator), due to the need to evaluate comparative efficacy and safety for single-arm trials. Unanchored matching-adjusted indirect comparison (MAIC) and unanchored simulated treatment comparison (STC) are currently the only two approaches available for population-adjusted indirect comparisons based on single-arm trials. However, there is a notable underutilisation of unanchored STC in HTA, largely due to a lack of understanding of its implementation. We therefore develop a novel way to implement unanchored STC by incorporating standardisation/marginalisation and the NORmal To Anything (NORTA) algorithm for sampling covariates. This methodology aims to derive a suitable marginal treatment effect without aggregation bias for HTA evaluations. We use a non-parametric bootstrap and propose separately calculating the standard error for the IPD study and the comparator study to ensure the appropriate quantification of the uncertainty associated with the estimated treatment effect. The performance of our proposed unanchored STC approach is evaluated through a comprehensive simulation study focused on binary outcomes. Our findings demonstrate that the proposed approach is asymptotically unbiased. We argue that unanchored STC should be considered when conducting unanchored indirect comparisons with single-arm studies, presenting a robust approach for HTA decision-making.

A methodology for projecting the return on investment of training technologies.

Each year significant tax dollars are spent on the development of new technologies to increase efficiency and/or reduce costs of military training. However, there are currently no validated methods or measures to quantify the return on investment for adopting these new technologies for military training. Estimating the return on investment (ROI) for training technology adoption involves 1) developing a methodology or framework, 2) validating measures and methods, and 3) assessing predictive validity. The current paper describes a projective methodology using the Kirkpatrick framework to compare projected tangible and intangible benefits against tangible and intangible costs to estimate future ROI. The use-case involved an advanced technology demonstration in which sixty aircrew participated in a series of live, virtual, and constructive (LVC) exercises over a five-week period. Participants evaluated the technology's potential costs and benefits according to the Kirkpatrick framework of training program evaluation, and analyses resulted in a nominal projection of $488 million dollars saved, significant enhancements in large-force proficiency, and 1.4 lives saved over ten years at an implementation rate of 0.5% of budgeted flight hours. A discussion of theoretical implications, data-based limitations, and recommendations for future research are provided.

Feasibility, safety, and patient acceptability of electronic inspiratory muscle training in patients who require prolonged mechanical ventilation in the intensive care unit: A dual-centre observational study.

BACKGROUND: Inspiratory muscle training (IMT) is an intervention that can be used to rehabilitate the respiratory muscle deconditioning experienced by patients with critical illness, requiring prolonged mechanical ventilation. Clinicians are currently using mechanical threshold IMT devices that have limited resistance ranges. OBJECTIVES: The objective of this study was to evaluate the safety, feasibility, and acceptability of using an electronic device to facilitate IMT with participants requiring prolonged mechanical ventilation. METHOD: A dual-centre observational cohort study, with convenience sampling, was conducted at two tertiary intensive care units. Daily training supervised by intensive care unit physiotherapists was completed with the electronic IMT device. A priori definitions for feasibility, safety, and acceptability were determined. Feasibility was defined as more than 80% of planned sessions completed. Safety was defined as no major adverse events and less than 3% minor adverse event rate, and acceptability was evaluated following the acceptability of intervention framework principles. RESULTS: Forty participants completed 197 electronic IMT treatment sessions. Electronic IMT was feasible, with 81% of planned sessions completed. There were 10% minor adverse events and no major adverse events. All the minor adverse events were transient without clinical consequences. All the participants who recalled completing electronic IMT sessions reported that the training was acceptable. Acceptability was demonstrated; over 85% of participants reported that electronic IMT was either helpful or beneficial and that electronic IMT assisted their recovery. CONCLUSION: Electronic IMT is feasible and acceptable to complete with critically ill participants who require prolonged mechanical ventilation. As all minor adverse events were transient without clinical consequences, electronic IMT can be considered a relatively safe intervention with patients who require prolonged mechanical ventilation.

Trajectories of Child Cognitive and Socioemotional Development and Associations with Adolescent Health in the UK Millennium Cohort Study.

OBJECTIVE: To identify and describe distinct trajectories of cognitive and socioemotional development during childhood and to examine their relationships with adolescent health. STUDY DESIGN: We used group-based multitrajectory modeling applied to longitudinal data on 11 564 children up to age 14 years from the UK Millennium Cohort study to identify trajectories of cognitive and socioemotional development measured using validated instruments. We assessed associations between the derived trajectories and baseline socioeconomic, parental, and school factors using multinomial regression. Logistic regression was used to assess associations between trajectory groups and adolescent health at age 14 and 17 years. RESULTS: Four child development trajectories were identified: "no problems" (76.5%); "late socio-emotional problems" (10.1%); "early cognitive and socioemotional problems" (8.6%); and "persistent cognitive and socioemotional problems" (4.8%). Those in the problem trajectories were more socioeconomically disadvantaged. Compared with the "no problem" trajectory, the "late socioemotional problems" trajectory had increased odds of overweight and mental ill-health at age 14 years of 1.50 (95% CI 1.24-1.81) and 2.51 (2.03-3.10), respectively. For the "persistent problems" group, the OR for overweight was 1.41 (1.04-1.91), and for mental ill-health, 3.01 (2.10-3.30). For both groups, the associations persisted to age 17 years. CONCLUSIONS: In a representative UK cohort, groups of distinct trajectories of cognitive and socioemotional development were identified. Adverse development, if unresolved, can have a negative impact on weight and mental health in adolescence. Socioemotional development was the main driver of the impact on adolescent health and this requires emphasis in child health policy.

Guidance on the use of complex systems models for economic evaluations of public health interventions.

To help health economic modelers respond to demands for greater use of complex systems models in public health. To propose identifiable features of such models and support researchers to plan public health modeling projects using these models. A working group of experts in complex systems modeling and economic evaluation was brought together to develop and jointly write guidance for the use of complex systems models for health economic analysis. The content of workshops was informed by a scoping review. A public health complex systems model for economic evaluation is defined as a quantitative, dynamic, non-linear model that incorporates feedback and interactions among model elements, in order to capture emergent outcomes and estimate health, economic and potentially other consequences to inform public policies. The guidance covers: when complex systems modeling is needed; principles for designing a complex systems model; and how to choose an appropriate modeling technique. This paper provides a definition to identify and characterize complex systems models for economic evaluations and proposes guidance on key aspects of the process for health economics analysis. This document will support the development of complex systems models, with impact on public health systems policy and decision making.

General-Purpose Methods for Simulating Survival Data for Expected Value of Sample Information Calculations.

BACKGROUND: Expected value of sample information (EVSI) quantifies the expected value to a decision maker of reducing uncertainty by collecting additional data. EVSI calculations require simulating plausible data sets, typically achieved by evaluating quantile functions at random uniform numbers using standard inverse transform sampling (ITS). This is straightforward when closed-form expressions for the quantile function are available, such as for standard parametric survival models, but these are often unavailable when assuming treatment effect waning and for flexible survival models. In these circumstances, the standard ITS method could be implemented by numerically evaluating the quantile functions at each iteration in a probabilistic analysis, but this greatly increases the computational burden. Thus, our study aims to develop general-purpose methods that standardize and reduce the computational burden of the EVSI data-simulation step for survival data. METHODS: We developed a discrete sampling method and an interpolated ITS method for simulating survival data from a probabilistic sample of survival probabilities over discrete time units. We compared the general-purpose and standard ITS methods using an illustrative partitioned survival model with and without adjustment for treatment effect waning. RESULTS: The discrete sampling and interpolated ITS methods agree closely with the standard ITS method, with the added benefit of a greatly reduced computational cost in the scenario with adjustment for treatment effect waning. CONCLUSIONS: We present general-purpose methods for simulating survival data from a probabilistic sample of survival probabilities that greatly reduce the computational burden of the EVSI data-simulation step when we assume treatment effect waning or use flexible survival models. The implementation of our data-simulation methods is identical across all possible survival models and can easily be automated from standard probabilistic decision analyses. HIGHLIGHTS: Expected value of sample information (EVSI) quantifies the expected value to a decision maker of reducing uncertainty through a given data collection exercise, such as a randomized clinical trial. In this article, we address the problem of computing EVSI when we assume treatment effect waning or use flexible survival models, by developing general-purpose methods that standardize and reduce the computational burden of the EVSI data-generation step for survival data.We developed 2 methods for simulating survival data from a probabilistic sample of survival probabilities over discrete time units, a discrete sampling method and an interpolated inverse transform sampling method, which can be combined with a recently proposed nonparametric EVSI method to accurately estimate EVSI for collecting survival data.Our general-purpose data-simulation methods greatly reduce the computational burden of the EVSI data-simulation step when we assume treatment effect waning or use flexible survival models. The implementation of our data-simulation methods is identical across all possible survival models and can therefore easily be automated from standard probabilistic decision analyses.

Cost-Effectiveness and Value of Information Analysis of an Ambient Intelligent Geriatric Management (AmbIGeM) System Compared to Usual Care to Prevent Falls in Older People in Hospitals.

BACKGROUND: The Ambient Intelligent Geriatric Management (AmbIGeM) system combines wearable sensors with artificial intelligence to trigger alerts to hospital staff before a fall. A clinical trial found no effect across a heterogenous population, but reported a reduction in the injurious falls rate in a post hoc analysis of patients on Geriatric Evaluation Management Unit (GEMU) wards. Cost-effectiveness and Value of Information (VoI) analyses of the AmbIGeM system in GEMU wards was undertaken. METHODS: An Australian health-care system perspective and 5-year time horizon were used for the cost-effectiveness analysis. Implementation costs, inpatient costs and falls data were collected. Injurious falls were defined as causing bruising, laceration, fracture, loss of consciousness, or if the patient reported persistent pain. To compare costs and outcomes, generalised linear regression models were used to adjust for baseline differences between the intervention and usual care groups. Bootstrapping was used to represent uncertainty. For the VoI analysis, 10,000 different sample sizes with randomly sampled values ranging from 1 to 50,000 were tested to estimate the optimal sample size of a new trial that maximised the Expected Net Benefits of Sampling. RESULTS: An adjusted 0.036 fewer injurious falls (adjusted rate ratio of 0.56) and AUD$4554 lower costs were seen in the intervention group. However, uncertainty that the intervention is cost effective for the prevention of an injurious fall was present at all monetary values of this effectiveness outcome. A new trial with a sample of 4376 patients was estimated to maximise the Expected Net Benefit of Sampling, generating a net benefit of AUD$186,632 at a benefit-to-cost ratio of 1.1. CONCLUSIONS: The benefits to cost ratio suggests that a new trial of the AmbIGeM system in GEMU wards may not be high-value compared to other potential trials, and that the system should be implemented. However, a broader analysis of options for preventing falls in GEMU is required to fully inform decision making. TRIAL REGISTRATION: Australian and New Zealand Clinical Trial Registry (ACTRN 12617000981325).

Value of Information Analysis in Models to Inform Health Policy.

Value of information (VoI) is a decision-theoretic approach to estimating the expected benefits from collecting further information of different kinds, in scientific problems based on combining one or more sources of data. VoI methods can assess the sensitivity of models to different sources of uncertainty and help to set priorities for further data collection. They have been widely applied in healthcare policy making, but the ideas are general to a range of evidence synthesis and decision problems. This article gives a broad overview of VoI methods, explaining the principles behind them, the range of problems that can be tackled with them, and how they can be implemented, and discusses the ongoing challenges in the area.

Simulating Study Data to Support Expected Value of Sample Information Calculations: A Tutorial.

The expected value of sample information (EVSI) can be used to prioritize avenues for future research and design studies that support medical decision making and offer value for money spent. EVSI is calculated based on 3 key elements. Two of these, a probabilistic model-based economic evaluation and updating model uncertainty based on simulated data, have been frequently discussed in the literature. By contrast, the third element, simulating data from the proposed studies, has received little attention. This tutorial contributes to bridging this gap by providing a step-by-step guide to simulating study data for EVSI calculations. We discuss a general-purpose algorithm for simulating data and demonstrate its use to simulate 3 different outcome types. We then discuss how to induce correlations in the generated data, how to adjust for common issues in study implementation such as missingness and censoring, and how individual patient data from previous studies can be leveraged to undertake EVSI calculations. For all examples, we provide comprehensive code written in the R language and, where possible, Excel spreadsheets in the supplementary materials. This tutorial facilitates practical EVSI calculations and allows EVSI to be used to prioritize research and design studies.

An Efficient Method for Computing Expected Value of Sample Information for Survival Data from an Ongoing Trial.

BACKGROUND: Decisions about new health technologies are increasingly being made while trials are still in an early stage, which may result in substantial uncertainty around key decision drivers such as estimates of life expectancy and time to disease progression. Additional data collection can reduce uncertainty, and its value can be quantified by computing the expected value of sample information (EVSI), which has typically been described in the context of designing a future trial. In this article, we develop new methods for computing the EVSI of extending an existing trial's follow-up, first for an assumed survival model and then extending to capture uncertainty about the true survival model. METHODS: We developed a nested Markov Chain Monte Carlo procedure and a nonparametric regression-based method. We compared the methods by computing single-model and model-averaged EVSI for collecting additional follow-up data in 2 synthetic case studies. RESULTS: There was good agreement between the 2 methods. The regression-based method was fast and straightforward to implement, and scales easily to include any number of candidate survival models in the model uncertainty case. The nested Monte Carlo procedure, on the other hand, was extremely computationally demanding when we included model uncertainty. CONCLUSIONS: We present a straightforward regression-based method for computing the EVSI of extending an existing trial's follow-up, both where a single known survival model is assumed and where we are uncertain about the true survival model. EVSI for ongoing trials can help decision makers determine whether early patient access to a new technology can be justified on the basis of the current evidence or whether more mature evidence is needed. HIGHLIGHTS: Decisions about new health technologies are increasingly being made while trials are still in an early stage, which may result in substantial uncertainty around key decision drivers such as estimates of life-expectancy and time to disease progression. Additional data collection can reduce uncertainty, and its value can be quantified by computing the expected value of sample information (EVSI), which has typically been described in the context of designing a future trial.In this article, we have developed new methods for computing the EVSI of extending a trial's follow-up, both where a single known survival model is assumed and where we are uncertain about the true survival model. We extend a previously described nonparametric regression-based method for computing EVSI, which we demonstrate in synthetic case studies is fast, straightforward to implement, and scales easily to include any number of candidate survival models in the EVSI calculations.The EVSI methods that we present in this article can quantify the need for collecting additional follow-up data before making an adoption decision given any decision-making context.

Relationships between Child Development at School Entry and Adolescent Health-A Participatory Systematic Review.

The relationship between child development and adolescent health, and how this may be modified by socio-economic conditions, is poorly understood. This limits cross-sector interventions to address adolescent health inequality. This review summarises evidence on the associations between child development at school starting age and subsequent health in adolescence and identifies factors affecting associations. We undertook a participatory systematic review, searching electronic databases (MEDLINE, PsycINFO, ASSIA and ERIC) for articles published between November 1990 and November 2020. Observational, intervention and review studies reporting a measure of child development and subsequent health outcomes, specifically weight and mental health, were included. Studies were individually and collectively assessed for quality using a comparative rating system of stronger, weaker, inconsistent or limited evidence. Associations between child development and adolescent health outcomes were assessed and reported by four domains of child development (socio-emotional, cognitive, language and communication, and physical development). A conceptual diagram, produced with stakeholders at the outset of the study, acted as a framework for narrative synthesis of factors that modify or mediate associations. Thirty-four studies were included. Analysis indicated stronger evidence of associations between measures of socio-emotional development and subsequent mental health and weight outcomes; in particular, positive associations between early externalising behaviours and later internalising and externalising, and negative associations between emotional wellbeing and later internalising and unhealthy weight. For all other domains of child development, although associations with subsequent health were positive, the evidence was either weaker, inconsistent or limited. There was limited evidence on factors that altered associations. Positive socio-emotional development at school starting age appears particularly important for subsequent mental health and weight in adolescence. More collaborative research across health and education is needed on other domains of development and on the mechanisms that link development and later health, and on how any relationship is modified by socio-economic context.

A Practical Guide to Modeling and Conducting a Cost-Effectiveness Analysis of Companion Biomarker Tests for Targeted Therapies Using R: Tutorial Paper.

Despite the increasing number of potential biomarkers identified in laboratories and reported in much literature, the adoption of biomarkers routinely available in clinical practice to inform treatment decisions is very limited. Reimbursement decisions for new health technologies are often informed by economic evaluations; however, economic evaluations of diagnostics/testing technologies, such as companion biomarker tests, are far less frequently reported than drugs. Furthermore, few countries provide the health economic evaluation methods guide specific to co-dependent technologies such as companion diagnostics or precision medicines. Therefore, this paper aims to guide the process of the development of cost-effectiveness models of cancer biomarkers for targeted therapies, focusing on companion diagnostics. This tutorial paper provides practical guidance on how to conduct economic evaluations of cancer biomarkers and how to model the characteristics of the biomarker tests as part of the value for money of corresponding targeted therapies. This paper presents a brief introduction to the methods and data requirements, a step-by-step guide to constructing a health economic model of companion cancer biomarkers, and a discussion of issues that arise in their application to healthcare decision making. This practical guidance is provided in R, and worked examples are provided in this paper with R codes in the accompanying electronic supplementary material.

Impact of child development at primary school entry on adolescent health-protocol for a participatory systematic review.

BACKGROUND: Reducing child health inequalities is a global health priority and evidence suggests that optimal development of knowledge, skills and attributes in early childhood could reduce health risks across the life course. Despite a strong policy rhetoric on giving children the 'best start in life', socioeconomic inequalities in children's development when they start school persist. So too do inequalities in child and adolescent health. These in turn influence health inequalities in adulthood. Understanding how developmental processes affect health in the context of socioeconomic factors as children age could inform a holistic policy approach to health and development from childhood through to adolescence. However, the relationship between child development and early adolescent health consequences is poorly understood. Therefore the aim of this review is to summarise evidence on the associations between child development at primary school starting age (3-7 years) and subsequent health in adolescence (8-15 years) and the factors that mediate or moderate this relationship. METHOD: A participatory systematic review method will be used. The search strategy will include; searches of electronic databases (MEDLINE, PsycINFO, ASSIA and ERIC) from November 1990 onwards, grey literature, reference searches and discussions with stakeholders. Articles will be screened using inclusion and exclusion criteria at title and abstract level, and at full article level. Observational, intervention and review studies reporting a measure of child development at the age of starting school and health outcomes in early adolescence, from a member country of the Organisation for Economic Co-operation and Development, will be included. The primary outcome will be health and wellbeing outcomes (such as weight, mental health, socio-emotional behaviour, dietary habits). Secondary outcomes will include educational outcomes. Studies will be assessed for quality using appropriate tools. A conceptual model, produced with stakeholders at the outset of the study, will act as a framework for extracting and analysing evidence. The model will be refined through analysis of the included literature. Narrative synthesis will be used to generate findings and produce a diagram of the relationship between child development and adolescent health. DISCUSSION: The review will elucidate how children's development at the age of starting school is related to subsequent health outcomes in contexts of socioeconomic inequality. This will inform ways to intervene to improve health and reduce health inequality in adolescents. The findings will generate knowledge of cross-sector relevance for health and education and promote inter-sectoral coherence in addressing health inequalities throughout childhood. PROTOCOL REGISTRATION: This systematic review protocol has been registered with PROSPERO CRD42020210011 .

A Review of Web-Based Tools for Value-of-Information Analysis.

Value-of-information analysis (VOI) is a decision-theoretic approach that is used to inform reimbursement decisions, optimise trial design and set research priorities. The application of VOI analysis for informing policy decisions in practice has been limited due, in part, to the perceived complexity associated with the calculation of VOI measures. Recent efforts have resulted in the development of efficient methods to estimate VOI measures and the development of user-friendly web-based tools to facilitate VOI calculations. We review the existing web-based tools including Sheffield Accelerated Value of Information (SAVI), the web interface to the BCEA (Bayesian Cost-Effectiveness Analysis) R package (BCEAweb), Rapid Assessment of Need for Evidence (RANE), and Value of Information for Cardiovascular Trials and Other Comparative Research (VICTOR). We describe what each tool is designed to do, the inputs they require, and the outputs they produce. Finally, we discuss how tools for VOI calculations might be improved in the future to facilitate the use of VOI analysis in practice.

Multiobjective Genetic Programming Can Improve the Explanatory Capabilities of Mechanism-Based Models of Social Systems.

The generative approach to social science, in which agent-based simulations (or other complex systems models) are executed to reproduce a known social phenomenon, is an important tool for realist explanation. However, a generative model, when suitably calibrated and validated using empirical data, represents just one viable candidate set of entities and mechanisms. The model only partially addresses the needs of an abductive reasoning process - specifically it does not provide insight into other viable sets of entities or mechanisms, nor suggest which of these are fundamentally constitutive for the phenomenon to exist. In this paper, we propose a new model discovery framework that more fully captures the needs of realist explanation. The framework exploits the implicit ontology of an existing human-built generative model to propose and test a plurality of new candidate model structures. Genetic programming is used to automate this search process. A multi-objective approach is used, which enables multiple perspectives on the value of any particular generative model - such as goodness-of-fit, parsimony, and interpretability - to be represented simultaneously. We demonstrate this new framework using a complex systems modeling case study of change and stasis in societal alcohol use patterns in the US over the period 1980-2010. The framework is successful in identifying three competing explanations of these alcohol use patterns, using novel integrations of social role theory not previously considered by the human modeler. Practitioners in complex systems modeling should use model discovery to improve the explanatory utility of the generative approach to realist social science.

A software architecture for mechanism-based social systems modelling in agent-based simulation models.

This paper introduces the MBSSM (Mechanism-Based Social Systems Modelling) software architecture that is designed for expressing mechanisms of social theories with individual behaviour components in a unified way and implementing these mechanisms in an agent-based simulation model. The MBSSM architecture is based on a middle-range theory approach most recently expounded by analytical sociology and is designed in the object-oriented programming paradigm with Unified Modelling Language diagrams. This paper presents two worked examples of using the architecture for modelling individual behaviour mechanisms that give rise to the dynamics of population-level alcohol use: a single-theory model of norm theory and a multi-theory model that combines norm theory with role theory. The MBSSM architecture provides a computational environment within which theories based on social mechanisms can be represented, compared, and integrated. The architecture plays a fundamental enabling role within a wider simulation model-based framework of abductive reasoning in which families of theories are tested for their ability to explain concrete social phenomena.

A Computationally Efficient Method for Probabilistic Parameter Threshold Analysis for Health Economic Evaluations.

Background. Threshold analysis is used to determine the threshold value of an input parameter at which a health care strategy becomes cost-effective. Typically, it is performed in a deterministic manner, in which inputs are varied one at a time while the remaining inputs are each fixed at their mean value. This approach will result in incorrect threshold values if the cost-effectiveness model is nonlinear or if inputs are correlated. Objective. To propose a probabilistic method for performing threshold analysis, which accounts for the joint uncertainty in all input parameters and makes no assumption about the linearity of the cost-effectiveness model. Methods. Three methods are compared: 1) deterministic threshold analysis (DTA); 2) a 2-level Monte Carlo approach, which is considered the gold standard; and 3) a regression-based method using a generalized additive model (GAM), which identifies threshold values directly from a probabilistic sensitivity analysis sample. Results. We applied the 3 methods to estimate the minimum probability of hospitalization for typhoid fever at which 3 different vaccination strategies become cost-effective in Uganda. The threshold probability of hospitalization at which routine vaccination at 9 months with catchup campaign to 5 years becomes cost-effective is estimated to be 0.060 and 0.061 (95% confidence interval [CI], 0.058-0.064), respectively, for 2-level and GAM. According to DTA, routine vaccination at 9 months with catchup campaign to 5 years would never become cost-effective. The threshold probability at which routine vaccination at 9 months with catchup campaign to 15 years becomes cost-effective is estimated to be 0.092 (DTA), 0.074 (2-level), and 0.072 (95% CI, 0.069-0.075) (GAM). GAM is 430 times faster than the 2-level approach. Conclusions. When the cost-effectiveness model is nonlinear, GAM provides similar threshold values to the 2-level Monte Carlo approach and is computationally more efficient. DTA provides incorrect results and should not be used.

Computing the Expected Value of Sample Information Efficiently: Practical Guidance and Recommendations for Four Model-Based Methods.

Value of information (VOI) analyses can help policy makers make informed decisions about whether to conduct and how to design future studies. Historically a computationally expensive method to compute the expected value of sample information (EVSI) restricted the use of VOI to simple decision models and study designs. Recently, 4 EVSI approximation methods have made such analyses more feasible and accessible. Members of the Collaborative Network for Value of Information (ConVOI) compared the inputs, the analyst's expertise and skills, and the software required for the 4 recently developed EVSI approximation methods. Our report provides practical guidance and recommendations to help inform the choice between the 4 efficient EVSI estimation methods. More specifically, this report provides: (1) a step-by-step guide to the methods' use, (2) the expertise and skills required to implement the methods, and (3) method recommendations based on the features of decision-analytic problems.