From Qualitative Research to Quantitative Preference Elicitation: An Example in Invasive Meningococcal Disease.

BACKGROUND: Qualitative research is fundamental for designing discrete choice experiments (DCEs) but is often underreported in the preference literature. We developed a DCE to elicit preferences for vaccination against invasive meningococcal disease (IMD) among adolescents and young people (AYP) and parents and legal guardians (PLG) in the United States. This article reports the targeted literature review and qualitative interviews that informed the DCE design and demonstrates how to apply the recent reporting guidelines for qualitative developmental work in preference studies. METHODS: This study included two parts: a targeted literature review and qualitative interviews. The Medline and Embase databases were searched for quantitative and qualitative studies on IMD and immunization. The results of the targeted literature review informed a qualitative interview guide. Sixty-minute, online, semi-structured interviews with AYP and PLG were used to identify themes related to willingness to be vaccinated against IMD. Participants were recruited through a third-party recruiter's database and commercial online panels. Interviews included vignettes about IMD and vaccinations and three thresholding exercises examining the effect of incidence rate, disability rate, and fatality rate on vaccination preferences. Participant responses related to the themes were counted. RESULTS: The targeted literature review identified 31 concepts that were synthesized into six topics for the qualitative interviews. Twenty AYP aged 16-23 years and 20 PLG of adolescents aged 11-17 years were interviewed. Four themes related to willingness to be vaccinated emerged: attitudes towards vaccination, knowledge and information, perception of IMD, and vaccine attributes. Most participants were concerned about IMD (AYP 60%; PLG 85%) and had positive views of vaccination (AYP 80%; PLG 60%). Ninety percent of AYP and 75% of PLG always chose vaccination over no vaccination, independent of IMD incidence rate, disability rate, or fatality rate. CONCLUSION: Willingness to be vaccinated against IMD was affected by vaccine attributes but largely insensitive to IMD incidence and severity. This article provides an example of how to apply the recent reporting guidelines for qualitative developmental work in preference studies, with 21 out of 22 items in the guidelines being considered.

Positioning whole exome sequencing in the diagnostic pathway for rare disease to optimise utility: a protocol for an observational cohort study and an economic evaluation.

INTRODUCTION: Despite the superior diagnostic performance of exome and genome sequencing compared with conventional genetic tests, evidence gaps related to clinical utility and cost effectiveness have limited their availability in routine clinical practice in many jurisdictions. To inform adoption and reimbursement policy, this protocol provides a chain of evidence approach to determining the diagnostic utility, clinical utility and cost-effectiveness of whole exome sequencing (WES) from seven medical genetic centres in two Canadian provinces. METHODS AND ANALYSIS: Using a multicentre observational cohort design, we will extract data specific to the pre-WES diagnostic pathway and 1-year post-WES medical management from electronic medical records for 650 patients with rare disease of suspected genetic aetiology who receive WES. The date from the clinical record will be linked to provincial administrative health database to capture healthcare resource use and estimate costs. Our analysis will: (1) define and describe diagnostic testing pathways that occur prior to WES among patients with rare disease, (2) determine the diagnostic utility of WES, characterised as the proportion of patients for whom causative DNA variants are identified, (3) determine the clinical utility of WES, characterised as a change in medical management triggered by WES results, (4) determine the pattern and cost of health service utilisation prior and 1 year following WES among patients who receive a diagnosis, do not receive a diagnosis, or receive an uncertain diagnosis and (5) estimate the cost-effectiveness of WES compared with conventional diagnostic testing pathways, measured by the incremental cost per additional patient diagnosed by WES using simulation modelling. ETHICS AND DISSEMINATION: This protocol was approved by Clinical Trials Ontario (CTO-1577) and research ethics boards at the University of Calgary (REB18-0744 and REB20-1449) and University of Alberta (Pro0009156). Findings will be disseminated through academic publications and policy reports.

The complexity of diagnosing rare disease: An organizing framework for outcomes research and health economics based on real-world evidence.

PURPOSE: To facilitate robust economic analyses of clinical exome and genome sequencing, this study was taken up with the objective of establishing a framework for organizing diagnostic testing trajectories for patients with rare disease. METHODS: We collected diagnostic investigations-related data before exome sequencing from the medical records of 228 cases. Medical geneticist experts participated in a consensus building process to develop the SOLVE Framework for organizing the complex range of observed tests. Experts categorized tests as indicator or nonindicator tests on the basis of their specificity for diagnosing rare diseases. Face validity was assessed using case vignettes. RESULTS: Most cases had symptom onset at birth (42.5%) or during childhood (43.4%) and had intellectual disability (73.3%). On average, the time spent seeking a diagnosis before sequencing was 1989 days (SD = 2137) and included 16 tests (SD = 14). Agreement across experts on test categories ranged from 83% to 96%. The SOLVE Framework comprised observed tests, including 186 indicator and 39 nonindicator tests across cytogenetic/molecular, biochemical, imaging, electrical, and pathology test categories. CONCLUSION: Real-world diagnostic testing data can be ascertained and organized to reflect the complexity of the journey of the patients with rare diseases. SOLVE Framework will improve the accuracy and certainty associated with value-based assessments of genomic sequencing.

Methods for Conducting Stated Preference Research with Children and Adolescents in Health: A Scoping Review of the Application of Discrete Choice Experiments.

BACKGROUND: Discrete choice experiments (DCEs) are a common method used to describe and quantitatively assess preferences in health applications. Increasingly, DCEs have been used to elicit preferences from children and adolescents and generate evidence to inform policies affecting this population. OBJECTIVES: The aim of this review was to summarize and describe the application of DCEs conducted with children and adolescents and describe author-reported age-specific considerations in design, implementation, and analysis. METHODS: A scoping review was conducted using a 'pearl-growing' technique whereby the reference lists of existing systematic reviews of DCEs were used to identify potential studies conducted with children or adolescents as respondents published between 1990 and 2017. This list was supplemented with an updated electronic search using the same strategy as the initial reviews to identify studies from 2017 to 2020. RESULTS: Of 480 studies identified, 19 were included; topics included vaccines (32%), drugs/medical devices (26%), treatment or health promotion interventions/programs (21%), warning labels on cigarettes/nicotine products (10%), and preferences for physical activity and healthy food choices (10%). The youngest reported age for independent DCE completion was 8 years. Approaches to assessing validity and reliability of choices were consistent with best practices for the conduct of DCEs. Reported age-specific considerations included use of visual aids, age-appropriate language, reducing task complexity and cognitive burden, and exploration of interpretation of willingness-to-pay. CONCLUSION: The number of DCEs conducted with children and adolescents has increased in recent years. Detailed explanation of why reported age-specific considerations were necessary, how they could be used to interpret results, or to understand the appropriateness of this methodology for different age groups was limited. Despite a recognition of the need for special consideration when conducting DCEs in this population, the unique issues in the context of age-specific considerations are largely unexplored, and further research is required. Moving forward, stated preference research conducted with children and adolescents should report in more detail methods of recruitment, results of validity assessments, and provide specific reflection on the extent to which modeled results are consistent with expectations and underlying theory.

Economic Evaluation of Public Health Interventions: An Application to Interventions for the Prevention of Violence Against Women and Girls Implemented by the "What Works to Prevent Violence Against Women and Girls?" Global Program.

Violence against women and girls (VAWG) has important social, economic, and public health impacts. Governments and international donors are increasing their investment in VAWG prevention programs, yet clear guidelines to assess the "value for money" of these interventions are lacking. Improved costing and economic evaluation of VAWG prevention can support programming through supporting priority setting, justifying investment, and planning the financing of VAWG prevention services. This article sets out a standardized methodology for the economic evaluation of complex, that is, multicomponent and/or multiplatform, programs designed to prevent VAWG in low- and middle-income countries (LMICs). It outlines an approach that can be used alongside the most recent guidance for the economic evaluation of public health interventions in LMICs. It defines standardized methods of data collection and analysis, outcomes, and unit costs (i.e., average costs per person reached, output or service delivered), and provides guidance to investigate the uncertainty in cost-effectiveness estimates and report results. The costing approach has been developed and piloted as part of the "What Works to Prevent Violence Against Women and Girls?" (What Works?) program in five countries. This article and its supplementary material can be used by both economists and non-economists to contribute to the generation of new cost-effectiveness data on VAWG prevention, and ultimately improve the allocative efficiency and financing across VAWG programs.

(Un)standardized testing: the diagnostic odyssey of children with rare genetic disorders in Alberta, Canada.

PURPOSE: We provide a description of the diagnostic odyssey for a cohort of children seeking diagnosis of a rare genetic disorder in terms of the time from initial consultation to most recent visit or receipt of diagnosis, the number of tests per patient, and the types of tests received. METHODS: Retrospective chart review of 299 children seen at the Alberta Children's Hospital (ACH) Genetics Clinic (GC) for whom the result of at least one single-gene test, gene panel, or chromosome microarray analysis (CMA) was recorded. RESULTS: Of 299 patients, 90 (30%) received a diagnosis in the period of the review. Patients had an average of 5.4 tests each; 236 (79%) patients received CMA; 172 (58%) patients received single-gene tests and 34 (11%) received gene panels; 167 (56%) underwent imaging/electrical activity studies. The mean observation period was 898 days (95% confidence interval [CI] 791, 1004). Among patients with visits recorded prior to visiting ACH GC, 43% of the total observation time occurred prior to the GC. CONCLUSION: As genomic technologies expand, the nature of the diagnostic odyssey will change. This study has outlined the current standard of care in the ACH GC, providing a baseline against which future changes can be assessed.

Attribution of Pediatric Acute Gastroenteritis Episodes and Emergency Department Visits to Norovirus Genogroups I and II.

BACKGROUND: Norovirus is a leading cause of acute gastroenteritis. With vaccines in development, population-based estimates of norovirus burden are needed to identify target populations, quantify potential benefits, and understand disease dynamics. METHODS: We estimated the attributable fraction (AF) for norovirus infections in children, defined as the proportion of children testing positive for norovirus whose gastroenteritis was attributable to norovirus. We calculated the standardized incidence and emergency department (ED) visit rates attributable to norovirus using provincial gastroenteritis visit administrative data. RESULTS: From 3731 gastroenteritis case patients and 2135 controls we determined that the AFs were 67.0% (95% confidence interval [CI], 31.5%-100%) and 91.6% (88.8%-94.4%) for norovirus genogroups I (GI) and II (GII), respectively. Norovirus GII AF varied by season but not age. We attributed 116 episodes (95% CI, 103-129) and 59 (51-67) ED visits per 10 000 child-years to norovirus GII across all ages, accounting for 20% and 18% of all medically attended gastroenteritis episodes and ED visits, respectively. CONCLUSIONS: In children, a large proportion of norovirus GII detections reflect causation, demonstrating significant potential for norovirus GII vaccines. Seasonal variation in the norovirus GII AF may have implications for understanding the role asymptomatic carriage plays in disease dynamics.

Economic evaluations of strategies to prevent sports and recreational injury among children and adolescents: a systematic review.

OBJECTIVE: To identify, summarise and critically assess studies reporting costs and consequences of sport and recreation injury prevention strategies among children and adolescents. DESIGN: Systematic review. METHODS AND DATA SOURCES: We searched MEDLINE (Ovid), EMBASE, CINAHL, Pubmed, Econlit and SPORTDiscus and PEDE. Included studies were peer reviewed full economic evaluations or cost analyses of sport/recreation injury prevention among children and adolescents≤18 years of age. The Pediatric Quality Appraisal Questionnaire was used for quality assessment. RESULTS: The initial search yielded 1896 unique records; eight studies met inclusion criteria. Six studies were related to injury prevention in the context of recreation, two were related to sports. For recreation studies in cycling and swimming: costs per head injury averted was US$3109 to $228 197; costs per hospitalisation avoided was US$3526 to 872 794; cost per life saved/death avoided was US$3531 to $103 518 154. Sport interventions in hockey and soccer were cost saving (fewer injuries and lower costs). Global quality assessments ranged from poor to good. Important limitations included short time horizons and intermediate outcome measures. CONCLUSIONS: Few rigorous economic evaluations related to sport and recreation injury prevention have been conducted. The range of estimates and variation in outcomes used preclude specific conclusions; however, where strategies both improve health and are cost saving, implementation should be prioritised. Future economic evaluations should incorporate time horizons sufficient to capture changes in long-term health and use utility-based outcome measures in order to capture individual preferences for changes in health states and facilitate comparison across intervention types.

Individual Preferences for Child and Adolescent Vaccine Attributes: A Systematic Review of the Stated Preference Literature.

BACKGROUND: Discrete choice experiments are increasingly used to assess preferences for vaccines and vaccine service delivery. OBJECTIVES: To synthesize and critically assess the application of discrete choice experiments in childhood/adolescent vaccines, to describe how discrete choice experiments have been applied to understand preferences, and to evaluate the use of discrete choice experiment data to inform estimates of vaccine uptake. METHODS: We conducted a systematic review of six electronic databases. Included studies were discrete choice experiments and conjoint analyses published from 2000 to 2016 related to childhood/adolescent vaccines where respondents were parents, children/adolescents, or service providers. Validity assessment was used to assess study quality and risk of bias. RESULTS: In total, 27 articles were included, representing 21 different studies. A majority of articles were published between 2011 and 2016. Vaccines studied included human papillomavirus (24%), influenza (19%), meningococcal vaccines (14%), childhood vaccines (14%), hypothetical vaccines (10%), hepatitis B (5%), and diphtheria, tetanus, pertussis, hepatitis B, poliomyelitis, and Haemophilus influenzae type b (5%). Most studies assessed parent preferences (67%). The most common attributes were risk (24%), degree/duration of protection (21%), and cost (15%). Commonly reported outcome measures were estimates of uptake (33%), willingness-to-pay (22%), and other marginal rates of substitution (14%). Validity assessments yielded high scores overall. Areas of weakness included low response rates, inefficient experimental design, and failure to conduct formative qualitative work and a pilot of the discrete choice experiment. CONCLUSION: This is the first systematic review of childhood/adolescent vaccine-related discrete choice experiments. In future, special attention should be paid to ensuring that choice context and discrete choice experiment design are compatible to generate reliable estimates of uptake.

Cost and cost-effectiveness analysis of a community mobilisation intervention to reduce intimate partner violence in Kampala, Uganda.

BACKGROUND: Intimate partner violence (IPV) poses a major public health concern. To date there are few rigorous economic evaluations of interventions aimed at preventing IPV in low-income settings. This study provides a cost and cost effectiveness analysis of SASA!, a community mobilisation intervention to change social norms and prevent IPV. METHODS: An economic evaluation alongside a cluster randomised controlled trial. Both financial and economic costs were collected retrospectively from the provider's perspective to generate total and unit cost estimates over four years of intervention programming. Univariate sensitivity analysis is conducted to estimate the impact of uncertainty in cost and outcome measures on results. RESULTS: The total cost of developing the SASA! Activist Kit is estimated as US$138,598. Total intervention costs over four years are estimated as US$553,252. The annual cost of supporting 351 activists to conduct SASA! activities was approximately US$389 per activist and the average cost per person reached in intervention communities was US$21 over the full course of the intervention, or US$5 annually. The primary trial outcome was past year experience of physical IPV with an estimated 1201 cases averted (90% CI: 97-2307 cases averted). The estimated cost per case of past year IPV averted was US$460. CONCLUSION: This study provides the first economic evaluation of a community mobilisation intervention aimed at preventing IPV. SASA! unit costs compare favourably with gender transformative interventions and support services for survivors of IPV. TRIAL REGISTRATION: ClinicalTrials.gov # NCT00790959.

Young People's Preferences for Family Planning Service Providers in Rural Malawi: A Discrete Choice Experiment.

OBJECTIVE: To quantify the impact of service provider characteristics on young people's choice of family planning (FP) service provider in rural Malawi in order to identify strategies for increasing access and uptake of FP among youth. METHODS AND FINDINGS: A discrete choice experiment was developed to assess the relative impact of service characteristics on preferences for FP service providers among young people (aged 15-24). Four alternative providers were included (government facility, private facility, outreach and community based distribution of FP) and described by six attributes (the distance between participants' home and the service delivery point, frequency of service delivery, waiting time at the facility, service providers' attitude, availability of FP commodities and price). A random parameters logit model was used to estimate preferences for service providers and the likely uptake of services following the expansion of outreach and community based distribution (CBDA) services. In the choice experiment young people were twice as likely to choose a friendly provider (government service odds ratio [OR] = 2.45, p<0.01; private service OR = 1.99, p<0.01; CBDA OR = 1.88, p<0.01) and more than two to three times more likely to choose a provider with an adequate supply of FP commodities (government service OR = 2.48, p<0.01; private service OR = 2.33, p<0.01; CBDA = 3.85, p<0.01). Uptake of community based services was greater than facility based services across a variety of simulated service scenarios indicating that such services may be an effective means of expanding access for youth in rural areas and an important tool for increasing service uptake among youth. CONCLUSIONS: Ensuring that services are acceptable to young people may require additional training for service providers in order to ensure that all providers are friendly and non-judgemental when dealing with younger clients and to ensure that supplies are consistently available.

Designing a package of sexual and reproductive health and HIV outreach services to meet the heterogeneous preferences of young people in Malawi: results from a discrete choice experiment.

BACKGROUND: This article examines young people's preferences for integrated family planning (FP) and HIV services in rural Malawi. Different hypothetical configurations for outreach services are presented using a Discrete Choice Experiment (DCE). Responses are analysed using Random Parameters Logit and Generalised Mixed Logit (GMXL) models in preference space and a GMXL model parameterised in willingness-to-pay space. Simulations are used to estimate the proportion of respondents expected to choose different service packages as elements are varied individually and in combination. RESULTS: Responses were collected from 537 young people aged 15-24. Results show that when considering attending an outreach service to access family planning young people value confidentiality and the availability of HIV services including HIV counselling and testing (HCT) and HIV treatment, though significant observable and unobservable heterogeneity is present. Female respondents and those aged 20-24 were less concerned with service confidentiality compared to male respondents and those aged 15-19; respondents who were in a relationship at the time of the survey valued confidentiality more than those who reported being single. The addition of sports and recreation for young people may also be an attractive feature of a youth-friendly service; however, preferences for this attribute vary according to respondent gender. Results of the simulation modelling indicate that the most preferred service package is one that offers confidential services, both HCT and HIV treatment and sports for youth, with up to 32% of respondents expected to choose this service over a service where clients may have concerns over confidentiality, only HCT is available and there are no additional activities for young people. Estimates of willingness-to-pay for service attributes indicate that respondents were willing to pay up to USD$1.76 for confidentiality, USD$0.65 for a service offering both HCT and HIV treatment and USD$0.26 for a service including sports for youth. CONCLUSIONS: Young people were able to complete a complex DCE and appeared to trade between the different characteristics used to describe the outreach services. These findings may offer important insight to policy makers designing youth friendly SRH outreach services and providers aiming to improve the acceptability and uptake of FP services.

The Costs of Delivering Integrated HIV and Sexual Reproductive Health Services in Limited Resource Settings.

OBJECTIVE: To present evidence on the total costs and unit costs of delivering six integrated sexual reproductive health and HIV services in a high and medium HIV prevalence setting, in order to support policy makers and planners scaling up these essential services. DESIGN: A retrospective facility based costing study conducted in 40 non-government organization and public health facilities in Kenya and Swaziland. METHODS: Economic and financial costs were collected retrospectively for the year 2010/11, from each study site with an aim to estimate the cost per visit of six integrated HIV and SRH services. A full cost analysis using a combination of bottom-up and step-down costing methods was conducted from the health provider's perspective. The main unit of analysis is the economic unit cost per visit for each service. Costs are converted to 2013 International dollars. RESULTS: The mean cost per visit for the HIV/SRH services ranged from $Int 14.23 (PNC visit) to $Int 74.21 (HIV treatment visit). We found considerable variation in the unit costs per visit across settings with family planning services exhibiting the least variation ($Int 6.71-52.24) and STI treatment and HIV treatment visits exhibiting the highest variation in unit cost ranging from ($Int 5.44-281.85) and ($Int 0.83-314.95), respectively. Unit costs of visits were driven by fixed costs while variability in visit costs across facilities was explained mainly by technology used and service maturity. CONCLUSION: For all services, variability in unit costs and cost components suggest that potential exists to reduce costs through better use of both human and capital resources, despite the high proportion of expenditure on drugs and medical supplies. Further work is required to explore the key drivers of efficiency and interventions that may facilitate efficiency improvements.

The impact of HIV/SRH service integration on workload: analysis from the Integra Initiative in two African settings.

BACKGROUND: There is growing interest in integration of HIV and sexual and reproductive health (SRH) services as a way to improve the efficiency of human resources (HR) for health in low- and middle-income countries. Although this is supported by a wealth of evidence on the acceptability and clinical effectiveness of service integration, there is little evidence on whether staff in general health services can easily absorb HIV services. METHODS: We conducted a descriptive analysis of HR integration through task shifting/sharing and staff workload in the context of the Integra Initiative - a large-scale five-year evaluation of HIV/SRH integration. We describe the level, characteristics and changes in HR integration in the context of wider efforts to integrate HIV/SRH, and explore the impact of HR integration on staff workload. RESULTS: Improvements in the range of services provided by staff (HR integration) were more likely to be achieved in facilities which also improved other elements of integration. While there was no overall relationship between integration and workload at the facility level, HIV/SRH integration may be most influential on staff workload for provider-initiated HIV testing and counselling (PITC) and postnatal care (PNC) services, particularly where HIV care and treatment services are being supported with extra SRH/HIV staffing. Our findings therefore suggest that there may be potential for further efficiency gains through integration, but overall the pace of improvement is slow. CONCLUSIONS: This descriptive analysis explores the effect of HIV/SRH integration on staff workload through economies of scale and scope in high- and medium-HIV prevalence settings. We find some evidence to suggest that there is potential to improve productivity through integration, but, at the same time, significant challenges are being faced, with the pace of productivity gain slow. We recommend that efforts to implement integration are assessed in the broader context of HR planning to ensure that neither staff nor patients are negatively impacted by integration policy.

Using decision mapping to inform the development of a stated choice survey to elicit youth preferences for sexual and reproductive health and HIV services in rural Malawi.

The process of designing and developing discrete choice experiments (DCEs) is often under reported. The need to adequately report the results of qualitative work used to identify attributes and levels used in a DCE is recognised. However, one area that has received relatively little attention is the exploration of the choice question of interest. This paper provides a case study of the process used to design a stated preference survey to assess youth preferences for integrated sexual and reproductive health (SRH) and HIV outreach services in Malawi. Development and design consisted of six distinct but overlapping and iterative stages. Stage one was a review of the literature. Stage two involved developing a decision map to conceptualise the choice processes involved. Stage three included twelve focus group discussions with young people aged 15-24 (n = 113) and three key informant interviews (n = 3) conducted in Ntcheu District, Malawi. Stage four involved analysis of qualitative data and identification of potential attributes and levels. The choice format and experimental design were selected in stages five and six. The results of the literature review were used to develop a decision map outlining the choices that young people accessing SRH services may face. For youth that would like to use services two key choices were identified: the choice between providers and the choice of service delivery attributes within a provider type. Youth preferences for provider type are best explored using a DCE with a labelled design, while preferences for service delivery attributes associated with a particular provider are better understood using an unlabelled design. Consequently, two DCEs were adopted to jointly assess preferences in this context. Used in combination, the results of the literature review, the decision mapping process and the qualitative work provided robust approach to designing the DCEs individually and as complementary pieces of work.