Methodological issues in economic evaluations of emergency transport systems in low-income and middle-income countries.

A recent systematic review identified few papers on the economic evaluation of systems for emergency transport of acutely ill or injured patients. In addition, we found no articles dealing with the methodological challenges posed by such studies in low-income or middle-income countries. We therefore carried out an analysis of issues that are of particular salience to this important topic. This is an intellectual study in which we develop models, identify their limitations, suggest potential extensions to the models and discuss priorities for empirical studies to populate models. First, we develop a general model to calculate changes in survival contingent on the reduced time to treatment that an emergency transport system is designed to achieve. Second, we develop a model to estimate transfer times over an area that will be served by a proposed transfer system. Third, we discuss difficulties in obtaining parameters with which to populate the models. Fourth, we discuss costs, both direct and indirect, of an emergency transfer service. Fifth, we discuss the issue that outcomes other than survival should be considered and that the effects of a service are a weighted sum over all the conditions and severities for which the service caters. Lastly, based on the above work, we identify priorities for research. To our knowledge, this is the first study to identify and frame issues in the health economics of acute transfer systems and to develop models to calculate survival rates from basic parameters, such as time delay/survival relationships, that vary by intervention type and context.

Changes in weekend and weekday care quality of emergency medical admissions to 20 hospitals in England during implementation of the 7-day services national health policy.

BACKGROUND: In 2013, the English National Health Service launched the policy of 7-day services to improve care quality and outcomes for weekend emergency admissions. AIMS: To determine whether the quality of care of emergency medical admissions is worse at weekends, and whether this has changed during implementation of 7-day services. METHODS: Using data from 20 acute hospital Trusts in England, we performed randomly selected structured case record reviews of patients admitted to hospital as emergencies at weekends and on weekdays between financial years 2012-2013 and 2016-2017. Senior doctor ('specialist') involvement was determined from annual point prevalence surveys. The primary outcome was the rate of clinical errors. Secondary outcomes included error-related adverse event rates, global quality of care and four indicators of good practice. RESULTS: Seventy-nine clinical reviewers reviewed 4000 admissions, 800 in duplicate. Errors, adverse events and care quality were not significantly different between weekend and weekday admissions, but all improved significantly between epochs, particularly errors most likely influenced by doctors (clinical assessment, diagnosis, treatment, prescribing and communication): error rate OR 0.78; 95% CI 0.70 to 0.87; adverse event OR 0.48, 95% CI 0.33 to 0.69; care quality OR 0.78, 95% CI 0.70 to 0.87; all adjusted for age, sex and ethnicity. Postadmission in-hospital care processes improved between epochs and were better for weekend admissions (vital signs with National Early Warning Score and timely specialist review). Preadmission processes in the community were suboptimal at weekends and deteriorated between epochs (fewer family doctor referrals, more patients with chronic disease or palliative care designation). CONCLUSIONS AND IMPLICATIONS: Hospital care quality of emergency medical admissions is not worse at weekends and has improved during implementation of the 7-day services policy. Causal pathways for the weekend effect may extend into the prehospital setting.

Sicker patients account for the weekend mortality effect among adult emergency admissions to a large hospital trust.

OBJECTIVE: To determine whether the higher weekend admission mortality risk is attributable to increased severity of illness. DESIGN: Retrospective analysis of 4 years weekend and weekday adult emergency admissions to a university teaching hospital in England. OUTCOME MEASURES: 30-day postadmission weekend:weekday mortality ratios adjusted for severity of illness (baseline National Early Warning Score (NEWS)), routes of admission to hospital, transfer to the intensive care unit (ICU) and demographics. RESULTS: Despite similar emergency department daily attendance rates, fewer patients were admitted on weekends (mean admission rate 91/day vs 120/day) because of fewer general practitioner referrals. Weekend admissions were sicker than weekday (mean NEWS 1.8 vs 1.7, p=0.008), more likely to undergo transfer to ICU within 24 hours (4.2% vs 3.0%), spent longer in hospital (median 3 days vs 2 days) and less likely to experience same-day discharge (17.2% vs 21.9%) (all p values <0.001).The crude 30-day postadmission mortality ratio for weekend admission (OR=1.13; 95% CI 1.08 to 1.19) was attenuated using standard adjustment (OR=1.11; 95% CI 1.05 to 1.17). In patients for whom NEWS values were available (90%), the crude OR (1.07; 95% CI 1.01 to 1.13) was not affected with standard adjustment. Adjustment using NEWS alone nullified the weekend effect (OR=1.02; 0.96-1.08).NEWS completion rates were higher on weekends (91.7%) than weekdays (89.5%). Missing NEWS was associated with direct transfer to intensive care bypassing electronic data capture. Missing NEWS in non-ICU weekend patients was associated with a higher mortality and fewer same-day discharges than weekdays. CONCLUSIONS: Patients admitted to hospital on weekends are sicker than those admitted on weekdays. The cause of the weekend effect may lie in community services.

Relative efficiency of unequal cluster sizes in stepped wedge and other trial designs under longitudinal or cross-sectional sampling.

BACKGROUND: A cluster trial with unequal cluster sizes often has lower precision than one with equal clusters, with a corresponding inflation of the design effect. For parallel group trials, adjustments to the design effect are available under sampling models with a single intracluster correlation. Design effects for equal clusters under more complex scenarios have appeared recently (including stepped wedge trials under cross-sectional or longitudinal sampling). We investigate the impact of unequal cluster size in these more general settings. RESULTS: Assuming a linear mixed model with an exchangeable correlation structure that incorporates cluster and subject autocorrelation, we compute the relative efficiency (RE) of a trial with clusters of unequal size under a size-stratified randomization scheme, as compared to an equal cluster trial with the same total number of observations. If there are no within-cluster time effects, the RE exceeds that for a parallel trial. In general, the RE is a weighted average of the RE for a parallel trial and the RE for a crossover trial in the same clusters. Existing approximations for parallel designs are extended to the general setting. Increasing the cluster size by the factor (1 + CV2 ), where CV is the coefficient of variation of cluster size, leads to conservative sample sizes, as in a popular method for parallel trials. CONCLUSION: Methods to assess experimental precision for single-period parallel trials with unequal cluster sizes can be extended to stepped wedge and other complete layouts under longitudinal or cross-sectional sampling. In practice, the loss of precision due to unequal cluster sizes is unlikely to exceed 12%.

Impact of a commercial order entry system on prescribing errors amenable to computerised decision support in the hospital setting: a prospective pre-post study.

BACKGROUND: In this UK study, we investigated the impact of computerised physician order entry (CPOE) and clinical decision support (CDS) implementation on the rate of 78 high-risk prescribing errors amenable to CDS. METHODS: We conducted a preintervention/postintervention study in three acute hospitals in England. A predefined list of prescribing errors was incorporated into an audit tool. At each site, approximately 4000 prescriptions were reviewed both pre-CPOE and 6 months post-CPOE implementation. The number of opportunities for error and the number of errors that occurred were collated. Error rates were then calculated and compared between periods, as well as by the level of CDS. RESULTS: The prescriptions of 1244 patients were audited pre-CPOE and 1178 post-CPOE implementation. A total of 28 526 prescriptions were reviewed, with 21 138 opportunities for error identified based on 78 defined errors. Across the three sites, for those prescriptions where opportunities for error were identified, the error rate was found to reduce significantly post-CPOE implementation, from 5.0% to 4.0% (P<0.001). CDS implementation by error type was found to differ significantly between sites, ranging from 0% to 88% across clinical contraindication, dose/frequency, drug interactions and other error types (P<0.001). Overall, 43/78 (55%) of the errors had some degree of CDS implemented in at least one of the hospitals. CONCLUSIONS: Implementation of CPOE with CDS was associated with clinically important reductions in the rate of high-risk prescribing errors. Given the pre-post design, these findings however need to be interpreted with caution. The occurrence of errors was found to be highly dependent on the level of restriction of CDS presented to the prescriber, with the effect that different configurations of the same CPOE system can produce very different results.

Reviewing deaths in British and US hospitals: a study of two scales for assessing preventability.

BACKGROUND: Standardised mortality ratios do not provide accurate measures of preventable mortality. This has generated interest in using case notes to assess the preventable component of mortality. But, different methods of measurement have not been compared. We compared the reliability of two scales for assessing preventability and the correspondence between them. METHODS: Medical specialists reviewed case notes of patients who had died in hospital, using two instruments: a five-point Likert scale and a continuous (0-100) scale of preventability. To enhance generalisability, we used two different hospital datasets with different types of acute medical patients across different epochs, and in two jurisdictions (UK and USA). We investigated the reliability of measurement and correspondence of preventability estimates across the two scales. Ordinal mixed effects regression methods were used to analyse the Likert scale and to calibrate it against the continuous scale. We report the estimates of the probability a death could have been prevented, accounting for reviewer inconsistency. RESULTS: Correspondence between the two scales was strong; the Likert categories explained most of the variation (76% UK, 73% USA) in the continuous scale. Measurement reliability was low, but similar across the two instruments in each dataset (intraclass correlation: 0.27, UK; 0.23, USA). Adjusting for the inconsistency of reviewer judgements reduced the proportion of cases with high preventability, such that the proportion of all deaths judged probably or definitely preventable on the balance of probability was less than 1%. CONCLUSIONS: The correspondence is high between a Likert and a continuous scale, although the low reliability of both would suggest careful measurement design would be needed to use either scale. Few to no cases are above the threshold when using a balance of probability approach to determining a preventable death, and in any case, there is little evidence supporting anything more than an ordinal correspondence between these reviewer estimates of probability and the true probability. Thus, it would be more defensible to use them as an ordinal measure of the quality of care received by patients who died in the hospital.

Statistical efficiency and optimal design for stepped cluster studies under linear mixed effects models.

In stepped cluster designs the intervention is introduced into some (or all) clusters at different times and persists until the end of the study. Instances include traditional parallel cluster designs and the more recent stepped-wedge designs. We consider the precision offered by such designs under mixed-effects models with fixed time and random subject and cluster effects (including interactions with time), and explore the optimal choice of uptake times. The results apply both to cross-sectional studies where new subjects are observed at each time-point, and longitudinal studies with repeat observations on the same subjects. The efficiency of the design is expressed in terms of a 'cluster-mean correlation' which carries information about the dependency-structure of the data, and two design coefficients which reflect the pattern of uptake-times. In cross-sectional studies the cluster-mean correlation combines information about the cluster-size and the intra-cluster correlation coefficient. A formula is given for the 'design effect' in both cross-sectional and longitudinal studies. An algorithm for optimising the choice of uptake times is described and specific results obtained for the best balanced stepped designs. In large studies we show that the best design is a hybrid mixture of parallel and stepped-wedge components, with the proportion of stepped wedge clusters equal to the cluster-mean correlation. The impact of prior uncertainty in the cluster-mean correlation is considered by simulation. Some specific hybrid designs are proposed for consideration when the cluster-mean correlation cannot be reliably estimated, using a minimax principle to ensure acceptable performance across the whole range of unknown values. © 2016 The Authors. Statistics in Medicine published by John Wiley & Sons Ltd.

Stepped-wedge cluster randomised controlled trials: a generic framework including parallel and multiple-level designs.

Stepped-wedge cluster randomised trials (SW-CRTs) are being used with increasing frequency in health service evaluation. Conventionally, these studies are cross-sectional in design with equally spaced steps, with an equal number of clusters randomised at each step and data collected at each and every step. Here we introduce several variations on this design and consider implications for power. One modification we consider is the incomplete cross-sectional SW-CRT, where the number of clusters varies at each step or where at some steps, for example, implementation or transition periods, data are not collected. We show that the parallel CRT with staggered but balanced randomisation can be considered a special case of the incomplete SW-CRT. As too can the parallel CRT with baseline measures. And we extend these designs to allow for multiple layers of clustering, for example, wards within a hospital. Building on results for complete designs, power and detectable difference are derived using a Wald test and obtaining the variance-covariance matrix of the treatment effect assuming a generalised linear mixed model. These variations are illustrated by several real examples. We recommend that whilst the impact of transition periods on power is likely to be small, where they are a feature of the design they should be incorporated. We also show examples in which the power of a SW-CRT increases as the intra-cluster correlation (ICC) increases and demonstrate that the impact of the ICC is likely to be smaller in a SW-CRT compared with a parallel CRT, especially where there are multiple levels of clustering. Finally, through this unified framework, the efficiency of the SW-CRT and the parallel CRT can be compared.

A multistudy analysis investigating systematic differences in cardiovascular trial results between Europe and Asia.

OBJECTIVE: To assess whether there are differences in the results of cardiovascular trials between Europe and Asia using data from an extensive collection of randomised controlled trials. STUDY DESIGN AND SETTING: All meta-analyses containing randomised controlled trials (RCT's) for the treatment or prevention of cardiovascular diseases were searched for in The Cochrane Library (2000-2008) and MEDLINE (2005-2008). Analysis was then conducted within and over each meta-analysis which satisfied given criteria. Separate estimates of treatment effect were calculated for Europe and Asia in each meta-analysis and then compared. Estimates of a common inter-continental difference over all meta-analyses were also calculated and meta-regression was performed. This was performed for both fatal and non-fatal end points. RESULTS: The literature search identified 59 meta-analyses that satisfied the inclusion criteria. After exclusion, the number of meta-analyses reporting greater effect sizes in Asia than in Europe was significantly more than would be expected by chance (fatal 12/14, p=0.013; non-fatal 23/32, p=0.020). CONCLUSIONS: This study provides some evidence that for cardiovascular interventions treatment effect estimation differs between Europe and Asia, with respect to both fatal and non-fatal end points.

Meta-analysis of randomized phase II trials to inform subsequent phase III decisions.

BACKGROUND: If multiple Phase II randomized trials exist then meta-analysis is favorable to increase statistical power and summarize the existing evidence about an intervention's effect in order to help inform Phase III decisions. We consider some statistical issues for meta-analysis of Phase II trials for this purpose, as motivated by a real example involving nine Phase II trials of bolus thrombolytic therapy in acute myocardial infarction with binary outcomes. METHODS: We propose that a Bayesian random effects logistic regression model is most suitable as it models the binomial distribution of the data, helps avoid continuity corrections, accounts for between-trial heterogeneity, and incorporates parameter uncertainty when making inferences. The model also allows predictions that inform Phase III decisions, and we show how to derive: (i) the probability that the intervention will be truly beneficial in a new trial, and (ii) the probability that, in a new trial with a given sample size, the 95% credible interval for the odds ratio will be entirely in favor of the intervention. As Phase II trials are potentially optimistic due to bias in design and reporting, we also discuss how skeptical prior distributions can reduce this optimism to make more realistic predictions. RESULTS: In the example, the model identifies heterogeneity in intervention effect missed by an I-squared of 0%. Prediction intervals accounting for this heterogeneity are shown to support subsequent Phase III trials. The probability of success in Phase III trials increases as the sample size increases, up to 0.82 for intracranial hemorrhage and 0.79 for reinfarction outcomes. CONCLUSIONS: The choice of meta-analysis methods can influence the decision about whether a trial should proceed to Phase III and thus need to be clearly documented and investigated whenever a Phase II meta-analysis is performed.

Protocol for evaluation of the cost-effectiveness of ePrescribing systems and candidate prototype for other related health information technologies.

BACKGROUND: This protocol concerns the assessment of cost-effectiveness of hospital health information technology (HIT) in four hospitals. Two of these hospitals are acquiring ePrescribing systems incorporating extensive decision support, while the other two will implement systems incorporating more basic clinical algorithms. Implementation of an ePrescribing system will have diffuse effects over myriad clinical processes, so the protocol has to deal with a large amount of information collected at various 'levels' across the system. METHODS/DESIGN: The method we propose is use of Bayesian ideas as a philosophical guide.Assessment of cost-effectiveness requires a number of parameters in order to measure incremental cost utility or benefit - the effectiveness of the intervention in reducing frequency of preventable adverse events; utilities for these adverse events; costs of HIT systems; and cost consequences of adverse events averted. There is no single end-point that adequately and unproblematically captures the effectiveness of the intervention; we therefore plan to observe changes in error rates and adverse events in four error categories (death, permanent disability, moderate disability, minimal effect). For each category we will elicit and pool subjective probability densities from experts for reductions in adverse events, resulting from deployment of the intervention in a hospital with extensive decision support. The experts will have been briefed with quantitative and qualitative data from the study and external data sources prior to elicitation. Following this, there will be a process of deliberative dialogues so that experts can "re-calibrate" their subjective probability estimates. The consolidated densities assembled from the repeat elicitation exercise will then be used to populate a health economic model, along with salient utilities. The credible limits from these densities can define thresholds for sensitivity analyses. DISCUSSION: The protocol we present here was designed for evaluation of ePrescribing systems. However, the methodology we propose could be used whenever research cannot provide a direct and unbiased measure of comparative effectiveness.

Effects of physical activity on the development and progression of microvascular complications in type 1 diabetes: retrospective analysis of the DCCT study.

BACKGROUND: To examine the effects of physical activity on the development and progression of microvascular complications in patients with type 1 diabetes. METHODS: A retrospective analysis of data from the Diabetes Control and Complications trial was undertaken. Physical activity data was collected at baseline for each of 1441 recruits, converted to metabolic equivalent of task values, and categorised according to the American College of Sports Medicine recommendations. The rates of development/progression of diabetic retinopathy, nephropathy and neuropathy were compared in those who achieved over twice recommended, up to twice recommended, and less than recommended metabolic equivalent of task levels of activity. The DCCT study had a mean duration of follow up of 6.5 years ending in 1993. RESULTS: A total of 271 subjects had a sustained three-step progression in diabetic retinopathy. The rates of development or progression of retinopathy showed no significant association with physical activity level. The number of outcomes for nephropathy and neuropathy were small and there was no significant association with physical activity level. CONCLUSIONS: We found no evidence that physical activity improves microvascular outcomes in type 1 diabetes. However we demonstrate no evidence of harm. We suggest that physical activity continues to play an important role in the management of type 1 diabetes.

What is the best strategy for investigating abnormal liver function tests in primary care? Implications from a prospective study.

OBJECTIVE: Evaluation of predictive value of liver function tests (LFTs) for the detection of liver-related disease in primary care. DESIGN: A prospective observational study. SETTING: 11 UK primary care practices. PARTICIPANTS: Patients (n=1290) with an abnormal eight-panel LFT (but no previously diagnosed liver disease). MAIN OUTCOME MEASURES: Patients were investigated by recording clinical features, and repeating LFTs, specific tests for individual liver diseases, and abdominal ultrasound scan. Patients were characterised as having: hepatocellular disease; biliary disease; tumours of the hepato-biliary system and none of the above. The relationship between LFT results and disease categories was evaluated by stepwise regression and logistic discrimination, with adjustment for demographic and clinical factors. True and False Positives generated by all possible LFT combinations were compared with a view towards optimising the choice of analytes in the routine LFT panel. RESULTS: Regression methods showed that alanine aminotransferase (ALT) was associated with hepatocellular disease (32 patients), while alkaline phosphatase (ALP) was associated with biliary disease (12 patients) and tumours of the hepatobiliary system (9 patients). A restricted panel of ALT and ALP was an efficient choice of analytes, comparing favourably with the complete panel of eight analytes, provided that 48 False Positives can be tolerated to obtain one additional True Positive. Repeating a complete panel in response to an abnormal reading is not the optimal strategy. CONCLUSIONS: The LFT panel can be restricted to ALT and ALP when the purpose of testing is to exclude liver disease in primary care.

Case-mix adjusted hospital mortality is a poor proxy for preventable mortality: a modelling study.

Risk-adjustment schemes are used to monitor hospital performance, on the assumption that excess mortality not explained by case mix is largely attributable to suboptimal care. We have developed a model to estimate the proportion of the variation in standardised mortality ratios (SMRs) that can be accounted for by variation in preventable mortality. The model was populated with values from the literature to estimate a predictive value of the SMR in this context-specifically the proportion of those hospitals with SMRs among the highest 2.5% that fall among the worst 2.5% for preventable mortality. The extent to which SMRs reflect preventable mortality rates is highly sensitive to the proportion of deaths that are preventable. If 6% of hospital deaths are preventable (as suggested by the literature), the predictive value of the SMR can be no greater than 9%. This value could rise to 30%, if 15% of deaths are preventable. The model offers a 'reality check' for case mix adjustment schemes designed to isolate the preventable component of any outcome rate.

Pricing of medical devices under coverage uncertainty--a modelling approach.

Product vendors and manufacturers are increasingly aware that purchasers of health care will fund new clinical treatments only if they are perceived to deliver value-for-money. This influences companies' internal commercial decisions, including the price they set for their products. Other things being equal, there is a price threshold, which is the maximum price at which the device will be funded and which, if its value were known, would play a central role in price determination. This paper examines the problem of pricing a medical device from the vendor's point of view in the presence of uncertainty about what the price threshold will be. A formal solution is obtained by maximising the expected value of the net revenue function, assuming a Bayesian prior distribution for the price threshold. A least admissible price is identified. The model can also be used as a tool for analysing proposed pricing policies when no formal prior specification of uncertainty is available.

Sample size calculations for cluster randomised controlled trials with a fixed number of clusters.

BACKGROUND: Cluster randomised controlled trials (CRCTs) are frequently used in health service evaluation. Assuming an average cluster size, required sample sizes are readily computed for both binary and continuous outcomes, by estimating a design effect or inflation factor. However, where the number of clusters are fixed in advance, but where it is possible to increase the number of individuals within each cluster, as is frequently the case in health service evaluation, sample size formulae have been less well studied. METHODS: We systematically outline sample size formulae (including required number of randomisation units, detectable difference and power) for CRCTs with a fixed number of clusters, to provide a concise summary for both binary and continuous outcomes. Extensions to the case of unequal cluster sizes are provided. RESULTS: For trials with a fixed number of equal sized clusters (k), the trial will be feasible provided the number of clusters is greater than the product of the number of individuals required under individual randomisation (nI) and the estimated intra-cluster correlation (ρ). So, a simple rule is that the number of clusters (k) will be sufficient provided: [formula in text]. Where this is not the case, investigators can determine the maximum available power to detect the pre-specified difference, or the minimum detectable difference under the pre-specified value for power. CONCLUSIONS: Designing a CRCT with a fixed number of clusters might mean that the study will not be feasible, leading to the notion of a minimum detectable difference (or a maximum achievable power), irrespective of how many individuals are included within each cluster.

Should patients with abnormal liver function tests in primary care be tested for chronic viral hepatitis: cost minimisation analysis based on a comprehensively tested cohort.

BACKGROUND: Liver function tests (LFTs) are ordered in large numbers in primary care, and the Birmingham and Lambeth Liver Evaluation Testing Strategies (BALLETS) study was set up to assess their usefulness in patients with no pre-existing or self-evident liver disease. All patients were tested for chronic viral hepatitis thereby providing an opportunity to compare various strategies for detection of this serious treatable disease. METHODS: This study uses data from the BALLETS cohort to compare various testing strategies for viral hepatitis in patients who had received an abnormal LFT result. The aim was to inform a strategy for identification of patients with chronic viral hepatitis. We used a cost-minimisation analysis to define a base case and then calculated the incremental cost per case detected to inform a strategy that could guide testing for chronic viral hepatitis. RESULTS: Of the 1,236 study patients with an abnormal LFT, 13 had chronic viral hepatitis (nine hepatitis B and four hepatitis C). The strategy advocated by the current guidelines (repeating the LFT with a view to testing for specific disease if it remained abnormal) was less efficient (more expensive per case detected) than a simple policy of testing all patients for viral hepatitis without repeating LFTs. A more selective strategy of viral testing all patients for viral hepatitis if they were born in countries where viral hepatitis was prevalent provided high efficiency with little loss of sensitivity. A notably high alanine aminotransferase (ALT) level (greater than twice the upper limit of normal) on the initial ALT test had high predictive value, but was insensitive, missing half the cases of viral infection. CONCLUSIONS: Based on this analysis and on widely accepted clinical principles, a "fast and frugal" heuristic was produced to guide general practitioners with respect to diagnosing cases of viral hepatitis in asymptomatic patients with abnormal LFTs. It recommends testing all patients where a clear clinical indication of infection is present (e.g. evidence of intravenous drug use), followed by testing all patients who originated from countries where viral hepatitis is prevalent, and finally testing those who have a notably raised ALT level (more than twice the upper limit of normal). Patients not picked up by this efficient algorithm had a risk of chronic viral hepatitis that is lower than the general population.

Integrating health economics into the product development cycle: a case study of absorbable pins for treating hallux valgus.

BACKGROUND: The probability of reimbursement is a key factor in determining whether to proceed with or abandon a product during its development. The purpose of this article is to illustrate how the methods of iterative Bayesian economic evaluation proposed in the literature can be incorporated into the development process of new medical devices, adapting them to face the relative scarcity of data and time that characterizes the process. METHODS: A 3-stage economic evaluation was applied: an early phase in which simple methods allow for a quick prioritization of competing products; a mid-stage in which developers synthesize the data into a decision model, identify the parameters for which more information is most valuable, and explore uncertainty; and a late stage, in which all relevant information is synthesized. A retrospective analysis was conducted of the case study of absorbable pins, compared with metallic fixation, in osteotomy to treat hallux valgus. RESULTS: The results from the early analysis suggest absorbable pins to be cost-effective under the beliefs and assumptions applied. The outputs from the models at the mid-stage analyses show the device to be cost-effective with a high probability. Late-stage analysis synthesizes evidence from a randomized controlled trial and informative priors, which are based on previous evidence. It also suggests that absorbable pins are the most cost-effective strategy, although the uncertainty in the model output increased considerably. CONCLUSIONS: This example illustrates how the method proposed allows decisions in the product development cycle to be based on the best knowledge that is available at each stage.