Validating health economic models with the Probabilistic Analysis Check dashBOARD (PACBOARD).

OBJECTIVES: Health economic (HE) models are often considered as "black boxes" because they are not publicly available and lack transparency, which prevents independent scrutiny of HE models. Additionally, validation efforts and validation status of HE models are not systematically reported. Methods to validate HE models in absence of their full underlying code are therefore urgently needed to improve health policy making.This study aimed to develop and test a generic dashboard to systematically explore the workings of HE models and validate their model parameters and outcomes. METHODS: The Probabilistic Analysis Check dashBOARD (PACBOARD) was developed using insights from literature, health economists, and a data scientist.Functionalities of PACBOARD are 1) exploring and validating model parameters and outcomes using standardised validation tests and interactive plots, 2) visualising and investigating the relationship between model parameters and outcomes using metamodelling, and 3) predicting health economic outcomes using the fitted metamodel.To test PACBOARD, two mock HE models were developed and errors were introduced in these models, e.g. negative costs inputs, utility values exceeding 1. PACBOARD metamodelling predictions of incremental net monetary benefit were validated against the original model's outcomes. RESULTS: PACBOARD automatically identified all errors introduced in the erroneous HE models. Metamodel predictions were accurate compared to the original model outcomes. CONCLUSIONS: PACBOARD is a unique dashboard aiming at improving the feasibility and transparency of validation efforts of HE models. PACBOARD allows users to explore the working of HE models using metamodelling based on HE models' parameters and outcomes.

A Blueprint for Multi-use Disease Modeling in Health Economics: Results from Two Expert-Panel Consultations.

BACKGROUND: The current use of health economic decision models in HTA is mostly confined to single use cases, which may be inefficient and result in little consistency over different treatment comparisons, and consequently inconsistent health policy decisions, for the same disorder. Multi-use disease models (MUDMs) (other terms: generic models, whole disease models, disease models) may offer a solution. However, much is uncertain about their definition and application. The current research aimed to develop a blueprint for the application of MUDMs. METHODS: We elicited expert opinion using a two-round modified Delphi process. The panel consisted of experts and stakeholders in health economic modelling from various professional backgrounds. The first questionnaire concerned definition, terminology, potential applications, issues and recommendations for MUDMs and was based on an exploratory scoping review. In the second round, the panel members were asked to reconsider their input, based on feedback regarding first-round results, and to score issues and recommendations for priority. Finally, adding input from external advisors and policy makers in a structured way, an overview of issues and challenges was developed during two team consensus meetings. RESULTS: In total, 54 respondents contributed to the panel results. The term 'multi-use disease models' was proposed and agreed upon, and a definition was provided. The panel prioritized 10 potential applications (with comparing alternative policies and supporting resource allocation decisions as the top 2), while 20 issues (with model transparency and stakeholders' roles as the top 2) were identified as challenges. Opinions on potential features concerning operationalization of multi-use models were given, with 11 of these subsequently receiving high priority scores (regular updates and revalidation after updates were the top 2). CONCLUSIONS: MUDMs would improve on current decision support regarding cost-effectiveness information. Given feasibility challenges, this would be most relevant for diseases with multiple treatments, large burden of disease and requiring more complex models. The current overview offers policy makers a starting point to organize the development, use, and maintenance of MUDMs and to support choices concerning which diseases and policy decisions they will be helpful for.

Cost-effectiveness of home-based screening of the general population for albuminuria to prevent progression of cardiovascular and kidney disease.

Background: Chronic kidney disease (CKD) is often detected late, leading to substantial health loss and high treatment costs. Screening the general population for albuminuria identifies individuals at high risk of kidney events and cardiovascular disease (CVD) who may benefit from early start of preventive interventions. Previous studies on the cost-effectiveness of albuminuria population screening were inconclusive, but were based on survey or cohort data rather than an implementation study, modelled screening as performed by general practitioners rather than home-based screening, and often included only benefits with respect to kidney events. We evaluated the cost-effectiveness of home-based general population screening for increased albuminuria based on real-world data obtained from a prospective implementation study taking into account prevention of CKD as well as CVD events. Methods: We developed an individual-level simulation model to compare home-based screening using a urine collection device with usual care (no home-based screening) in individuals of the general population aged 45-80, based on the THOMAS study (Towards HOMe-based Albuminuria Screening). Cost-effectiveness was assessed from the Dutch healthcare perspective with a lifetime horizon. The costs of the screening process and benefits of preventing CKD progression (dialysis and kidney transplantation) and CVD events (non-fatal myocardial infarction, non-fatal stroke, fatal CVD event) were reflected. Albuminuria detection led to treatment of identified risk factors. The model subsequently simulated CKD progression, the occurrence of CVD events, and death. The risks of experiencing CVD events were calculated using the SCORE2 CKD risk prediction model and individual-level data from the THOMAS study. Relative treatment effectiveness, quality of life scores, resource use, and cost inputs were obtained from literature. Model outcomes were the number of CKD and CVD-related events, total costs, quality-adjusted life years (QALYs), and the incremental cost-effectiveness ratio (ICER) per QALY gained by screening versus usual care. All results were obtained through probabilistic analysis. Findings: The absolute difference between screening versus usual care in lifetime probability of dialysis, kidney transplantation, non-fatal myocardial infarction, non-fatal stroke, and fatal CVD events were 0.2%, 0.05%, 0.6%, 0.6%, and 0.2%, respectively. This led to relative decreases compared to usual care in lifetime incidence of these events of 10.7%, 11.1%, 5.1%, 4.1%, and 1.6%, respectively. The incremental costs and QALYs of screening were €1607 and 0.17 QALY, respectively, which led to a corresponding ICER of €9225/QALY. The probability of screening being cost-effective for the Dutch willingness-to-pay threshold for preventive population screening of €20,000/QALY was 95.0%. Implementing the screening in the subgroup of 45-64 years old reduced the ICER (€7946/QALY), whereas implementing screening in the subgroup of 65-80 years old increased the ICER (€10,310/QALY). A scenario analysis assuming treatment optimization in all individuals with newly diagnosed risk factors or known risk factors not within target range reduced the ICER to €7083/QALY, resulting from the incremental costs and QALY gain of €2145 and 0.30, respectively. Interpretation: Home-based screening for increased albuminuria to prevent CVD and CKD events is likely cost-effective. More health benefits can be obtained by screening younger individuals and better optimization of care in individuals identified with newly diagnosed or known risk factors outside target range. Funding: Dutch Kidney Foundation, Top Sector Life Sciences & Health of the Dutch Ministry of Economic Affairs.

Introducing Open Science in Teaching Health Economic Modelling.

Open Science is gaining ground in all research fields, including health economics and outcomes research (HEOR). However, teaching Open Science is still in its infancy. This paper describes the design, implementation and evaluation of a teaching activity focusing on introducing Open Science during a Master's course during which participants have to develop a discrete event simulation. The teaching activity was organised as a series of lectures introducing different aspects of the Open Science philosophy and practices, such as good software coding practices, version control systems and reproducible research. The participants' increase in Open Science knowledge was elicited through a survey before and after the teaching innovation. After the teaching innovation, participants' knowledge of Open Science increased and they reported an improvement in Open Science-related skills, such as using a script-based statistical software, identifying and re-using open data, and collaborative script development. During the evaluation at the end of the course, the course participants mentioned that the Open Science-related content was interesting but would fit better within a course in which broader research-related content is taught. Based on this feedback, we will most likely narrow the scope of the Open-Science-related content in this course to Open Source Modelling which may better fit the scope of the course. This paper contains links to the teaching activities we developed and other resources which may be used to design teaching activities on Open Science. Herewith, we hope to inspire other teachers in including Open Science into their teaching.

Consolidated Health Economic Evaluation Reporting Standards - Value of Information (CHEERS-VOI): Explanation and Elaboration.

OBJECTIVES: Although the ISPOR Value of Information (VOI) Task Force's reports outline VOI concepts and provide good-practice recommendations, there is no guidance for reporting VOI analyses. VOI analyses are usually performed alongside economic evaluations for which the Consolidated Health Economic Evaluation Reporting Standards (CHEERS) 2022 Statement provides reporting guidelines. Thus, we developed the CHEERS-VOI checklist to provide reporting guidance and checklist to support the transparent, reproducible, and high-quality reporting of VOI analyses. METHODS: A comprehensive literature review generated a list of 26 candidate reporting items. These candidate items underwent a Delphi procedure with Delphi participants through 3 survey rounds. Participants rated each item on a 9-point Likert scale to indicate its relevance when reporting the minimal, essential information about VOI methods and provided comments. The Delphi results were reviewed at 2-day consensus meetings and the checklist was finalized using anonymous voting. RESULTS: We had 30, 25, and 24 Delphi respondents in rounds 1, 2, and 3, respectively. After incorporating revisions recommended by the Delphi participants, all 26 candidate items proceeded to the 2-day consensus meetings. The final CHEERS-VOI checklist includes all CHEERS items, but 7 items require elaboration when reporting VOI. Further, 6 new items were added to report information relevant only to VOI (eg, VOI methods applied). CONCLUSIONS: The CHEERS-VOI checklist should be used when a VOI analysis is performed alongside economic evaluations. The CHEERS-VOI checklist will help decision makers, analysts and peer reviewers in the assessment and interpretation of VOI analyses and thereby increase transparency and rigor in decision making.

Health Economic Research Assessing the Value of Early Detection of Cardiovascular Disease: A Systematic Review.

INTRODUCTION: Cardiovascular disease (CVD) is the most prominent cause of death worldwide and has a major impact on healthcare budgets. While early detection strategies may reduce the overall CVD burden through earlier treatment, it is unclear which strategies are (most) efficient. AIM: This systematic review reports on the cost effectiveness of recent early detection strategies for CVD in adult populations at risk. METHODS: PubMed and Scopus were searched to identify scientific articles published between January 2016 and May 2022. The first reviewer screened all articles, a second reviewer independently assessed a random 10% sample of the articles for validation. Discrepancies were solved through discussion, involving a third reviewer if necessary. All costs were converted to 2021 euros. Reporting quality of all studies was assessed using the CHEERS 2022 checklist. RESULTS: In total, 49 out of 5552 articles were included for data extraction and assessment of reporting quality, reporting on 48 unique early detection strategies. Early detection of atrial fibrillation in asymptomatic patients was most frequently studied (n = 15) followed by abdominal aortic aneurysm (n = 8), hypertension (n = 7) and predicted 10-year CVD risk (n = 5). Overall, 43 strategies (87.8%) were reported as cost effective and 11 (22.5%) CVD-related strategies reported cost reductions. Reporting quality ranged between 25 and 86%. CONCLUSIONS: Current evidence suggests that early CVD detection strategies are predominantly cost effective and may reduce CVD-related costs compared with no early detection. However, the lack of standardisation complicates the comparison of cost-effectiveness outcomes between studies. Real-world cost effectiveness of early CVD detection strategies will depend on the target country and local context. REGISTRATION OF SYSTEMATIC REVIEW: CRD42022321585 in International Prospective Registry of Ongoing Systematic Reviews (PROSPERO) submitted at 10 May 2022.

Population-Based Screening Using Low-Dose Chest Computed Tomography: A Systematic Review of Health Economic Evaluations.

BACKGROUND: Chest low-dose computed tomography (LDCT) is a promising technology for population-based screening because it is non-invasive, relatively inexpensive, associated with low radiation and highly sensitive to lung cancer. To improve the cost-effectiveness of lung cancer screening, simultaneous screening for other diseases could be considered. This systematic review was conducted to analyse studies that published evidence on the cost-effectiveness of chest LDCT screening programs for different diseases. METHODS: Scopus and PubMed were searched for English publications (1 January 2011-22 July 2022) using search terms related to screening, computed tomography and cost-effectiveness. An additional search specifically searched for the cost-effectiveness of screening for lung cancer, chronic obstructive pulmonary disease or cardiovascular disease. Included publications should present a full health economic evaluation of population screening with chest LDCT. The extracted data included the disease screened for, model type, country context of screening, inclusion of comorbidities or incidental findings, incremental costs, incremental effects and the resulting cost-effectiveness ratio amongst others. Reporting quality was assessed using the 2022 Consolidated Health Economic Evaluation Reporting Standards (CHEERS) checklist. RESULTS: The search yielded 1799 unique papers, of which 43 were included. Most papers focused on lung cancer screening (n = 40), and three were on coronary calcium scoring. Microsimulation was the most commonly applied modelling type (n = 16), followed by life table analysis (n = 10) and Markov cohort models (n = 10). Studies reflected the healthcare context of the US (n = 15), Canada (n = 4), the UK (n = 3) and 13 other countries. The reported incremental cost-effectiveness ratio ranged from US$10,000 to US$90,000/quality-adjusted life year (QALY) for lung cancer screening compared to no screening and was US$15,900/QALY-US$45,300/QALY for coronary calcium scoring compared to no screening. DISCUSSION: Almost all health economic evaluations of LDCT screening focused on lung cancer. Literature regarding the health economic benefits of simultaneous LDCT screening for multiple diseases is absent. Most studies suggest LDCT screening is cost-effective for current and former smokers aged 55-74 with a minimum of 30 pack-years of smoking history. Consequently, more evidence on LDCT is needed to support further cost-effectiveness analyses. Preferably evidence on simultaneous screening for multiple diseases is needed, but alternatively, on single-disease screening. REGISTRATION OF SYSTEMATIC REVIEW: Prospective Register of Ongoing Systematic Reviews registration CRD42021290228 can be accessed https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=290228 .

Drivers of healthcare costs in patients with ischaemic stroke: a hospital-based retrospective cohort study.

OBJECTIVES: To evaluate how costs of healthcare can be reduced, there is an increasing need to gain insight into the main drivers of such costs. We evaluated drivers of costs of predefined subgroups of patients who had a stroke by linking cost registration with clinical data. METHODS: We retrospectively selected 555 consecutive patients with ischaemic stroke participating between June 2011 and December 2016 in the Dutch Parelsnoer Initiative. Patient characteristics and costs of healthcare activities during hospital admission and the first 3 months after discharge were linked. Patients were divided in subgroups based on age, severity of stroke, stroke subtype, discharge destination and functional outcome. Unit cost per healthcare activity was based on 2018 rates for mutual service in euros. Mean total costs per subgroup were calculated. Multivariate analysis was performed to identify factors associated with costs. RESULTS: Number of admitted days was the main driver of total hospital costs (range 82%-93%) in all predefined subgroups of patients. Second driver was radiological diagnostic investigations (range 2%-9%). Highest costs were observed in patients with a younger age at the time of admission, a higher modified Rankin Scale at the time of discharge and a nursing home as discharge destination. The distribution of costs over the different healthcare activities was associated with stroke subtype; for example, in patients with a cardiac embolism most costs were spent on cardiology-related healthcare activities. CONCLUSION: The number of admitted days was the most important driver of costs in all subgroups of patients with ischaemic stroke. This implicates that to reduce healthcare costs for patients who had a stroke, focus should be on reducing length of hospital stay.

The societal impact of implementing an at-home blood sampling device for chronic care patients: patient preferences and cost impact.

BACKGROUND: Diabetes mellitus, cardiovascular diseases, chronic kidney disease, and thyroid diseases are chronic diseases that require regular monitoring through blood tests. This paper first investigates the experiences of chronic care patients with venipuncture and their expectations of an at-home blood-sampling device, and then assesses the impact on societal costs of implementing such a device in current practice. METHODS: An online survey was distributed among chronic care patients to gain insight into their experience of blood sampling in current practice, and their expectations of an at-home blood-sampling device. The survey results were used as input parameters in a patient-level monte carlo analysis developed to represent a hypothetical cohort of Dutch chronically ill patients to investigate the impact on societal costs compared to usual care. RESULTS: In total, 1311 patients participated in the survey, of which 31% experience the time spent on the phlebotomy appointment as a burden. Of all respondents, 71% prefer to use an at-home blood-sampling device to monitor their chronic disease. The cost analysis indicated that implementing an at-home blood-sampling device increases the cost of phlebotomy itself by €27.25 per patient per year, but it reduces the overall societal costs by €24.86 per patient per year, mainly due to limiting productivity loss. CONCLUSIONS: Patients consider an at-home blood-sampling device to be more user-friendly than venous phlebotomy on location. Long waiting times and crowded locations can be avoided by using an at-home blood-sampling device. Implementing such a device is likely cost-saving as it is expected to reduce societal costs.

Developing a dynamic simulation model to support the nationwide implementation of whole genome sequencing in lung cancer.

BACKGROUND: This study shows how dynamic simulation modeling can be applied in the context of the nationwide implementation of Whole Genome Sequencing (WGS) for non-small cell lung cancer (NSCLC) to inform organizational decisions regarding the use of complex and disruptive health technologies and how these decisions affect their potential value. METHODS: Using the case of the nationwide implementation of WGS into clinical practice in lung cancer in the Dutch healthcare system, we developed a simulation model to show that including service delivery features across the diagnostic pathway can provide essential insight into the affordability and accessibility of care at the systems level. The model was implemented as a hybrid Agent-Based Model and Discrete-Event Simulation model in AnyLogic and included 78 hospital agents, 7 molecular tumor board agents, 1 WGS facility agent, and 5313 patient agents each year in simulation time. RESULTS: The model included patient and provider heterogeneity, including referral patterns, capacity constraints, and diagnostic workflows. Patient preference and adoption by healthcare professionals were also modeled. The model was used to analyze a scenario in which only academic hospitals have implemented WGS. To prevent delays in the diagnostic pathway, the capacity to sequence at least 1600 biopsies yearly should be present. There is a two-fold increase in mean diagnostic pathway duration between no patients referred or all patients referred for further diagnostics. CONCLUSIONS: The systems model can complement conventional health economic evaluations to investigate how the organization of the workflow can influence the actual use and impact of WGS. Insufficient capacity to provide WGS and referral patterns can substantially impact the duration of the diagnostic pathway and thus should be considered in the implementation of WGS.

Systematic Review of Health Economic Evaluations Focused on Artificial Intelligence in Healthcare: The Tortoise and the Cheetah.

OBJECTIVES: This study aimed to systematically review recent health economic evaluations (HEEs) of artificial intelligence (AI) applications in healthcare. The aim was to discuss pertinent methods, reporting quality and challenges for future implementation of AI in healthcare, and additionally advise future HEEs. METHODS: A systematic literature review was conducted in 2 databases (PubMed and Scopus) for articles published in the last 5 years. Two reviewers performed independent screening, full-text inclusion, data extraction, and appraisal. The Consolidated Health Economic Evaluation Reporting Standards and Philips checklist were used for the quality assessment of included studies. RESULTS: A total of 884 unique studies were identified; 20 were included for full-text review, covering a wide range of medical specialties and care pathway phases. The most commonly evaluated type of AI was automated medical image analysis models (n = 9, 45%). The prevailing health economic analysis was cost minimization (n = 8, 40%) with the costs saved per case as preferred outcome measure. A total of 9 studies (45%) reported model-based HEEs, 4 of which applied a time horizon >1 year. The evidence supporting the chosen analytical methods, assessment of uncertainty, and model structures was underreported. The reporting quality of the articles was moderate as on average studies reported on 66% of Consolidated Health Economic Evaluation Reporting Standards items. CONCLUSIONS: HEEs of AI in healthcare are limited and often focus on costs rather than health impact. Surprisingly, model-based long-term evaluations are just as uncommon as model-based short-term evaluations. Consequently, insight into the actual benefits offered by AI is lagging behind current technological developments.

Efficacy, cost-minimization, and budget impact of a personalized discharge letter for basal cell carcinoma patients to reduce low-value follow-up care.

BACKGROUND: The incidence of keratinocyte carcinomas is high and rapidly growing. Approximately 80% of keratinocyte carcinomas consist of basal cell carcinomas (BCC) with 50% of these being considered as low-risk tumors. Nevertheless, 83% of the low-risk BCC patients were found to receive more follow-up care than recommended according to the Dutch BCC guideline, which is one visit post-treatment for this group. More efficient management could reduce unnecessary follow-up care and related costs. OBJECTIVES: To study the efficacy, cost-utility, and budget impact of a personalized discharge letter for low-risk BCC patients compared with usual care (no personalized letter). METHODS: In a multi-center intervention study, a personalized discharge letter in addition to usual care was compared to usual care in first-time BCC patients. Model-based cost-utility and budget impact analyses were conducted, using individual patient data gathered via surveys. The outcome measures were number of follow-up visits, costs and quality adjusted life years (QALY) per patient. RESULTS: A total of 473 first-time BCC patients were recruited. The personalized discharge letter decreased the number of follow-up visits by 14.8% in the first year. The incremental costs after five years were -€24.45 per patient. The QALYs were 4.12 after five years and very similar in both groups. The national budget impact was -€2,7 million after five years. CONCLUSIONS: The distribution of a personalized discharge letter decreases the number of unnecessary follow-up visits and implementing the intervention in a large eligible population would results in substantial cost savings, contributing to restraining the growing BCC costs.

Comparing Modeling Approaches for Discrete Event Simulations With Competing Risks Based on Censored Individual Patient Data: A Simulation Study and Illustration in Colorectal Cancer.

OBJECTIVES: This study aimed to provide detailed guidance on modeling approaches for implementing competing events in discrete event simulations based on censored individual patient data (IPD). METHODS: The event-specific distributions (ESDs) approach sampled times from event-specific time-to-event distributions and simulated the first event to occur. The unimodal distribution and regression approach sampled a time from a combined unimodal time-to-event distribution, representing all events, and used a (multinomial) logistic regression model to select the event to be simulated. A simulation study assessed performance in terms of relative absolute event incidence difference and relative entropy of time-to-event distributions for different types and levels of right censoring, numbers of events, distribution overlap, and sample sizes. Differences in cost-effectiveness estimates were illustrated in a colorectal cancer case study. RESULTS: Increased levels of censoring negatively affected the modeling approaches' performance. A lower number of competing events and higher overlap of distributions improved performance. When IPD were censored at random times, ESD performed best. When censoring occurred owing to a maximum follow-up time for 2 events, ESD performed better for a low level of censoring (ie, 10%). For 3 or 4 competing events, ESD better represented the probabilities of events, whereas unimodal distribution and regression better represented the time to events. Differences in cost-effectiveness estimates, both compared with no censoring and between approaches, increased with increasing censoring levels. CONCLUSIONS: Modelers should be aware of the different modeling approaches available and that selection between approaches may be informed by data characteristics. Performing and reporting extensive validation efforts remains essential to ensure IPD are appropriately represented.

Can we increase efficiency of CT lung cancer screening by combining with CVD and COPD screening? Results of an early economic evaluation.

OBJECTIVES: Estimating the maximum acceptable cost (MAC) per screened individual for low-dose computed tomography (LDCT) lung cancer (LC) screening, and determining the effect of additionally screening for chronic obstructive pulmonary disease (COPD), cardiovascular disease (CVD), or both on the MAC. METHODS: A model-based early health technology assessment (HTA) was conducted to estimate whether a new intervention could be cost-effective by calculating the MAC at a willingness-to-pay (WTP) of €20k/quality-adjusted life-year (QALY) and €80k/QALY, for a population of current and former smokers, aged 50-75 years in The Netherlands. The MAC was estimated based on incremental QALYs gained from a stage shift assuming screened individuals are detected in earlier disease stages. Data were obtained from literature and publicly available statistics and validated with experts. RESULTS: The MAC per individual for implementing LC screening at a WTP of €20k/QALY was €113. If COPD, CVD, or both were included in screening, the MAC increased to €230, €895, or €971 respectively. Scenario analyses assessed whether screening-specific disease high-risk populations would improve cost-effectiveness, showing that high-risk CVD populations were more likely to improve economic viability compared to COPD. CONCLUSIONS: The economic viability of combined screening is substantially larger than for LC screening alone, primarily due to benefits from CVD screening, and is dependent on the target screening population, which is key to optimise the screening program. The total cost of breast and cervical cancer screening is lower (€420) than the MAC of Big-3, indicating that Big-3 screening may be acceptable from a health economic perspective. KEY POINTS: • Once-off combined low-dose CT screening for lung cancer, COPD, and CVD in individuals aged 50-75 years is potentially cost-effective if screening would cost less than €971 per screened individual. • Multi-disease screening requires detailed insight into the co-occurrence of these diseases to identify the optimal target screening population. • With the same target screening population and WTP, lung cancer-only screening should cost less than €113 per screened individual to be cost-effective.

Choosing a Metamodel of a Simulation Model for Uncertainty Quantification.

BACKGROUND: Metamodeling may substantially reduce the computational expense of individual-level state transition simulation models (IL-STM) for calibration, uncertainty quantification, and health policy evaluation. However, because of the lack of guidance and readily available computer code, metamodels are still not widely used in health economics and public health. In this study, we provide guidance on how to choose a metamodel for uncertainty quantification. METHODS: We built a simulation study to evaluate the prediction accuracy and computational expense of metamodels for uncertainty quantification using life-years gained (LYG) by treatment as the IL-STM outcome. We analyzed how metamodel accuracy changes with the characteristics of the simulation model using a linear model (LM), Gaussian process regression (GP), generalized additive models (GAMs), and artificial neural networks (ANNs). Finally, we tested these metamodels in a case study consisting of a probabilistic analysis of a lung cancer IL-STM. RESULTS: In a scenario with low uncertainty in model parameters (i.e., small confidence interval), sufficient numbers of simulated life histories, and simulation model runs, commonly used metamodels (LM, ANNs, GAMs, and GP) have similar, good accuracy, with errors smaller than 1% for predicting LYG. With a higher level of uncertainty in model parameters, the prediction accuracy of GP and ANN is superior to LM. In the case study, we found that in the worst case, the best metamodel had an error of about 2.1%. CONCLUSION: To obtain good prediction accuracy, in an efficient way, we recommend starting with LM, and if the resulting accuracy is insufficient, we recommend trying ANNs and eventually also GP regression.

Evaluation of Real-World Healthcare Resource Utilization and Associated Costs in Children with Juvenile Idiopathic Arthritis: A Canadian Retrospective Cohort Study.

INTRODUCTION: Juvenile idiopathic arthritis (JIA) is a chronic rheumatic disease, whose multifaceted care path can lead to significant expenditure for the healthcare system. We aim to assess the real-world healthcare resource use (HCRU) and associated cost for children with JIA in a single center in Canada. METHODS: A single-center consecutive cohort of newly diagnosed patients with JIA attending the pediatric rheumatology clinic from 2011 to 2019 was identified using an administrative data algorithm and electronic medical charts. HCRU was estimated from six administrative health databases that included hospital admissions, emergency, outpatient care, practitioners' visits, medication, and laboratory and imaging tests. Costs were assigned using appropriate sources. We reported the yearly overall and JIA-associated HCRU and costs 5 years prior to and 6 years after the first visit to the pediatric rheumatologist. The Zhao and Tian estimator was used to calculate cumulative mean costs over a 6-year timeframe. Results were stratified by disease subtype. RESULTS: A total of 389 patients were identified. The yearly total overall mean costs per patient ranged between $804 and $4460 during the 5 years prior to the first visit to the pediatric rheumatologist and $8529 and $10,651 for the 6 years after. Medication cost, driven by use of biologic therapies, and outpatient visits were the greatest contributor to the total cost. The overall cumulative mean cost for 6 years of care was $48,649 per patient, while the JIA-associated cumulative mean cost was $26,820 per patient. During the first year of rheumatology care, systemic onset JIA had the highest cumulative mean overall cost, while oligoarticular JIA had the lowest cumulative mean cost. CONCLUSION: The care pathway for children with JIA can be expensive, and complex-and varies by JIA subtype. Although the yearly total mean cost per patient was constant, the distribution of costs changes over time with the introduction of biologic therapies later in the care pathway. This study provides a better understanding of the JIA costs profile and can help inform future economic studies.

Whole genome sequencing in oncology: using scenario drafting to explore future developments.

BACKGROUND: In oncology, Whole Genome Sequencing (WGS) is not yet widely implemented due to uncertainties such as the required infrastructure and expertise, costs and reimbursements, and unknown pan-cancer clinical utility. Therefore, this study aimed to investigate possible future developments facilitating or impeding the use of WGS as a molecular diagnostic in oncology through scenario drafting. METHODS: A four-step process was adopted for scenario drafting. First, the literature was searched for barriers and facilitators related to the implementation of WGS. Second, they were prioritized by international experts, and third, combined into coherent scenarios. Fourth, the scenarios were implemented in an online survey and their likelihood of taking place within 5 years was elicited from another group of experts. Based on the minimum, maximum, and most likely (mode) parameters, individual Program Evaluation and Review Technique (PERT) probability density functions were determined. Subsequently, individual opinions were aggregated by performing unweighted linear pooling, from which summary statistics were extracted and reported. RESULTS: Sixty-two unique barriers and facilitators were extracted from 70 articles. Price, clinical utility, and turnaround time of WGS were ranked as the most important aspects. Nine scenarios were developed and scored on likelihood by 18 experts. The scenario about introducing WGS as a clinical diagnostic with a lower price, shorter turnaround time, and improved degree of actionability, scored the highest likelihood (median: 68.3%). Scenarios with low likelihoods and strong consensus were about better treatment responses to more actionable targets (26.1%), and the effect of centralizing WGS (24.1%). CONCLUSIONS: Based on current expert opinions, the implementation of WGS as a clinical diagnostic in oncology is heavily dependent on the price, clinical utility (both in terms of identifying actionable targets as in adding sufficient value in subsequent treatment), and turnaround time. These aspects and the optimal way of service provision are the main drivers for the implementation of WGS and should be focused on in further research. More knowledge regarding these factors is needed to inform strategic decision making regarding the implementation of WGS, which warrants support from all relevant stakeholders.

Using Metamodeling to Identify the Optimal Strategy for Colorectal Cancer Screening.

OBJECTIVES: Metamodeling can address computational challenges within decision-analytic modeling studies evaluating many strategies. This article illustrates the value of metamodeling for evaluating colorectal cancer screening strategies while accounting for colonoscopy capacity constraints. METHODS: In a traditional approach, the best screening strategy was identified from a limited subset of strategies evaluated with the validated Adenoma and Serrated pathway to Colorectal CAncer model. In a metamodeling approach, metamodels were fitted to this limited subset to evaluate all potentially plausible strategies and determine the best overall screening strategy. Approaches were compared based on the best screening strategy in life-years gained compared with no screening. Metamodel runtime and accuracy was assessed. RESULTS: The metamodeling approach evaluated >40 000 strategies in <1 minute with high accuracy after 1 adaptive sampling step (mean absolute error: 0.0002 life-years) using 300 samples in total (generation time: 8 days). Findings indicated that health outcomes could be improved without requiring additional colonoscopy capacity. Obtaining similar insights using the traditional approach could require at least 1000 samples (generation time: 28 days). Suggested benefits from screening at ages <40 years require adequate validation of the underlying Adenoma and Serrated pathway to Colorectal CAncer model before making policy recommendations. CONCLUSIONS: Metamodeling allows rapid assessment of a vast set of strategies, which may lead to identification of more favorable strategies compared to a traditional approach. Nevertheless, metamodel validation and identifying extrapolation beyond the support of the original decision-analytic model are critical to the interpretation of results. The screening strategies identified with metamodeling support ongoing discussions on decreasing the starting age of colorectal cancer screening.