Independent Registries Are Cost-Effective Tools to Provide Mandatory Postauthorization Surveillance for Orphan Medicinal Products.

OBJECTIVES: Orphan medicinal products (OMPs) often receive market authorization under conditions imposed by regulators for ongoing postauthorization surveillance (PAS) to answer questions that remain at the time of market entry. This surveillance may be provided through industry-funded registries (IFRs). Nevertheless, data in these registries may not be of sufficient quality to answer these questions and may not always be accessible for regulatory review. We propose that a mandatory independent registry is an efficient and cost-effective tool for PAS for OMPs. METHODS: Using data from the Canadian Fabry Disease Initiative, we reviewed costs per unique patient from sites participating in both the independent national registry and IFRs for Fabry disease and compared data completeness from the Canadian Fabry Disease Initiative to that in published documents from IFRs. RESULTS: The costs of data collection through the independent registry were 17% to 36% (depending on site) lower than costs to collect data in the IFRs, and completeness of data collected through the independent registry was higher than that through the IFRs. Data from the independent registry were reviewed annually to guide indications for publicly funded Fabry disease therapy. Even when enrollment ceased to be a requirement to receive therapy, 77% of patients continued to enroll in the registry, suggesting the structure was acceptable to patients. CONCLUSIONS: Independent registries are cost-effective and efficient tools and should be mandated by regulatory agencies as the preferred tool for PAS for OMPs. Countries with publicly funded health systems should consider investment in registry infrastructure for OMPs.

Prioritizing Additional Data Collection to Reduce Decision Uncertainty in the HIV/AIDS Response in 6 US Cities: A Value of Information Analysis.

OBJECTIVES: The ambitious goals of the US Ending the HIV Epidemic initiative will require a targeted, context-specific public health response. Model-based economic evaluation provides useful guidance for decision making while characterizing decision uncertainty. We aim to quantify the value of eliminating uncertainty about different parameters in selecting combination implementation strategies to reduce the public health burden of HIV/AIDS in 6 US cities and identify future data collection priorities. METHODS: We used a dynamic compartmental HIV transmission model developed for 6 US cities to evaluate the cost-effectiveness of a range of combination implementation strategies. Using a metamodeling approach with nonparametric and deep learning methods, we calculated the expected value of perfect information, representing the maximum value of further research to eliminate decision uncertainty, and the expected value of partial perfect information for key groups of parameters that would be collected together in practice. RESULTS: The population expected value of perfect information ranged from $59 683 (Miami) to $54 108 679 (Los Angeles). The rank ordering of expected value of partial perfect information on key groups of parameters were largely consistent across cities and highest for parameters pertaining to HIV risk behaviors, probability of HIV transmission, health service engagement, HIV-related mortality, health utility weights, and healthcare costs. Los Angeles was an exception, where parameters on retention in pre-exposure prophylaxis ranked highest in contributing to decision uncertainty. CONCLUSIONS: Funding additional data collection on HIV/AIDS may be warranted in Baltimore, Los Angeles, and New York City. Value of information analysis should be embedded into decision-making processes on funding future research and public health intervention.

Evaluating the effectiveness, efficiency, cost and value of contacting study authors in a systematic review: a case study and worked example.

BACKGROUND: Studies find that identifying additional study data is possible by contacting study authors or experts. What is less certain is the time taken, costs involved and value found by using this supplementary search method. The purpose of this study is to determine the effectiveness, efficiency, cost and value of contacting study authors by e-mail, updating the evidence available for this search method. METHODS: Eighty-eight study authors, whose studies met title/abstract inclusion in a. systematic review, were contacted by e-mail. * effectiveness was assessed by comparing the number of study authors contacted. compared to the number of replies received; * efficiency was assessed by recording the time taken to contact study authors; * cost was assessed by comparing the efficiency of contacting authors with the. effectiveness; and * value was assessed by reading and comparing the published studies with the replies received to see if any unique data was identified. RESULTS: Contacting study authors took 6 h, 54 min and 25 s across 7 weeks. 38 answers (46%) were received from 83 possible contacts. Contacting study authors cost £80.33 or £2.11 per reply. We identified unique data from author replies when compared with data reported in published studies, determining this method as 'valuable'. CONCLUSIONS: Whilst our effectiveness findings differ from other studies, we believe that this study demonstrates the effectiveness of contacting study authors. By linking effectiveness to value and cost, we offer a new way to interpret the 'effectiveness' of this supplementary search method.

Challenges in Collecting Big Data in A Clinical Environment with Vulnerable Population: Lessons Learned from A Study Using A Multi-modal Sensors Platform.

Agitation is one of the most common behavioural and psychological symptoms in people living with dementia (PLwD). This behaviour can cause tremendous stress and anxiety on family caregivers and healthcare providers. Direct observation of PLwD is the traditional way to measure episodes of agitation. However, this method is subjective, bias-prone and timeconsuming. Importantly, it does not predict the onset of the agitation. Therefore, there is a need to develop a continuous monitoring system that can detect and/or predict the onset of agitation. In this study, a multi-modal sensor platform with video cameras, motion and door sensors, wristbands and pressure mats were set up in a hospital-based dementia behavioural care unit to develop a predictive system to identify the onset of agitation. The research team faced several barriers in the development and initiation of the study, namely addressing concerns about the study ethics, logistics and costs of study activities, device design for PLwD and limitations of its use in the hospital. In this paper, the strategies and methodologies that were implemented to address these challenges are discussed for consideration by future researchers who will conduct similar studies in a hospital setting.

Citizens unite for computational immunology!

Recruiting volunteers who can provide computational time, programming expertise, or puzzle-solving talent has emerged as a powerful tool for biomedical research. Recent projects demonstrate the potential for such 'crowdsourcing' efforts in immunology. Tools for developing applications, new funding opportunities, and an eager public make crowdsourcing a serious option for creative solutions for computationally-challenging problems. Expanded uses of crowdsourcing in immunology will allow for more efficient large-scale data collection and analysis. It will also involve, inspire, educate, and engage the public in a variety of meaningful ways. The benefits are real - it is time to jump in!

Collecting verbal autopsies: improving and streamlining data collection processes using electronic tablets.

BACKGROUND: There is increasing interest in using verbal autopsy to produce nationally representative population-level estimates of causes of death. However, the burden of processing a large quantity of surveys collected with paper and pencil has been a barrier to scaling up verbal autopsy surveillance. Direct electronic data capture has been used in other large-scale surveys and can be used in verbal autopsy as well, to reduce time and cost of going from collected data to actionable information. METHODS: We collected verbal autopsy interviews using paper and pencil and using electronic tablets at two sites, and measured the cost and time required to process the surveys for analysis. From these cost and time data, we extrapolated costs associated with conducting large-scale surveillance with verbal autopsy. RESULTS: We found that the median time between data collection and data entry for surveys collected on paper and pencil was approximately 3 months. For surveys collected on electronic tablets, this was less than 2 days. For small-scale surveys, we found that the upfront costs of purchasing electronic tablets was the primary cost and resulted in a higher total cost. For large-scale surveys, the costs associated with data entry exceeded the cost of the tablets, so electronic data capture provides both a quicker and cheaper method of data collection. CONCLUSIONS: As countries increase verbal autopsy surveillance, it is important to consider the best way to design sustainable systems for data collection. Electronic data capture has the potential to greatly reduce the time and costs associated with data collection. For long-term, large-scale surveillance required by national vital statistical systems, electronic data capture reduces costs and allows data to be available sooner.

Comparing time and motion methods to study personnel time in the context of a family planning supply chain intervention in Senegal.

BACKGROUND: A family planning (FP) supply chain intervention was introduced in Senegal in 2012 to reduce contraceptive stock-outs. Labour is the highest cost in low- and middle-income country supply chains. In this paper, we (1) understand time use of personnel working in the FP supply chain at health facilities in Senegal, (2) estimate the validity of self-administered timesheets (STs) relative to continuous observations (COs), and (3) describe the cost of data collection for each method. METHODS: We collected time use data for seven stockroom managers in six facilities using both ST and CO. Activities were categorized as follows: stock management associated with FP, non-FP stock management, other productive activities, non-productive activities, and waiting time. Paired t tests were used to compare the mean differences between the two methods in all categories and in productive time alone. RESULTS: Among all activities, the absolute and relative time spent on productive activities was higher when estimated by ST compared to CO. Conversely, waiting time was underestimated by STs. There was no difference in the relative time spent on non-productive activities. When comparing the distribution of the three productive activity categories, we found no evidence of a difference in relative time percentage estimates between CO and ST (FP stockroom management - 3.0%, 95% CI - 7.4 to 1.4%; non-FP stockroom management 3.4%, 95% CI - 2.8 to 9.6%; and other productive activities - 0.1%, 95% CI - 6.3 to 6.0%). Data collection costs for CO are 140% more than ST. CONCLUSION: STs were not a reliable method for measuring absolute labour time at health facilities in Senegal due to considerable underestimates of time waiting for clients. However, ST had acceptable reliability when examining distribution of productive time. Although CO provides more accurate absolute time estimates, the unit costs for data collection using this method are more than triple those for STs in Senegal.

Balancing the Optimal and the Feasible: A Practical Guide for Setting Up Patient Registries for the Collection of Real-World Data for Health Care Decision Making Based on Dutch Experiences.

OBJECTIVES: The aim of this article was to provide practical guidance in setting up patient registries to facilitate real-world data collection for health care decision making. METHODS: This guidance was based on our experiences and involvement in setting up patient registries in oncology in the Netherlands. All aspects were structured according to 1) mission and goals ("the Why"), 2) stakeholders and funding ("the Who"), 3) type and content ("the What"), and 4) identification and recruitment of patients, data handling, and pharmacovigilance ("the How"). RESULTS: The mission of most patient registries is improving patient health by improving the quality of patient care; monitoring and evaluating patient care is often the primary goal ("the Why"). It is important to align the objectives of the registry and agree on a clear and functional governance structure with all stakeholders ("the Who"). There is often a trade off between reliability, validity, and specificity of data elements and feasibility of data collection ("the What"). Patient privacy should be carefully protected, and address (inter-)national and local regulations. Patient registries can reveal unique safety information, but it can be challenging to comply with pharmacovigilance guidelines ("the How"). CONCLUSIONS: It is crucial to set up an efficient patient registry that serves its aims by collecting the right data of the right patient in the right way. It can be expected that patient registries will become the new standard alongside randomized controlled trials due to their unique value.

Modified ground-truthing: an accurate and cost-effective food environment validation method for town and rural areas.

BACKGROUND: A major concern in food environment research is the lack of accuracy in commercial business listings of food stores, which are convenient and commonly used. Accuracy concerns may be particularly pronounced in rural areas. Ground-truthing or on-site verification has been deemed the necessary standard to validate business listings, but researchers perceive this process to be costly and time-consuming. This study calculated the accuracy and cost of ground-truthing three town/rural areas in Minnesota, USA (an area of 564 miles, or 908 km), and simulated a modified validation process to increase efficiency without comprising accuracy. For traditional ground-truthing, all streets in the study area were driven, while the route and geographic coordinates of food stores were recorded. RESULTS: The process required 1510 miles (2430 km) of driving and 114 staff hours. The ground-truthed list of stores was compared with commercial business listings, which had an average positive predictive value (PPV) of 0.57 and sensitivity of 0.62 across the three sites. Using observations from the field, a modified process was proposed in which only the streets located within central commercial clusters (the 1/8 mile or 200 m buffer around any cluster of 2 stores) would be validated. Modified ground-truthing would have yielded an estimated PPV of 1.00 and sensitivity of 0.95, and would have resulted in a reduction in approximately 88 % of the mileage costs. CONCLUSIONS: We conclude that ground-truthing is necessary in town/rural settings. The modified ground-truthing process, with excellent accuracy at a fraction of the costs, suggests a new standard and warrants further evaluation.

Methodological Issues to Consider When Collecting Data to Estimate Poverty Impact in Economic Evaluations in Low-income and Middle-income Countries.

Out-of-pocket spending is increasingly recognized as an important barrier to accessing health care, particularly in low-income and middle-income countries (LMICs) where a large portion of health expenditure comes from out-of-pocket payments. Emerging universal healthcare policies prioritize reduction of poverty impact such as catastrophic and impoverishing healthcare expenditure. Poverty impact is therefore increasingly evaluated alongside and within economic evaluations to estimate the impact of specific health interventions on poverty. However, data collection for these metrics can be challenging in intervention-based contexts in LMICs because of study design and practical limitations. Using a set of case studies, this letter identifies methodological challenges in collecting patient cost data in LMIC contexts. These components are presented in a framework to encourage researchers to consider the implications of differing approaches in data collection and to report their approach in a standardized and transparent way.

Feasibility of Report Cards for Measuring Anesthesiologist Quality for Cardiac Surgery.

BACKGROUND: In creating the Merit-Based Incentive Payment System, Congress has mandated pay-for-performance (P4P) for all physicians, including anesthesiologists. There are currently no National Quality Forum-endorsed risk-adjusted outcome metrics for anesthesiologists to use as the basis for P4P. METHODS: Using clinical data from the New York State Cardiac Surgery Reporting System, we conducted a retrospective observational study of 55,436 patients undergoing cardiac surgery between 2009 and 2012. Hierarchical logistic regression modeling was used to examine the variation in in-hospital mortality or major complications (Q-wave myocardial infarction, renal failure, stroke, and respiratory failure) among anesthesiologists, controlling for patient demographics, severity of disease, comorbidities, and hospital quality. RESULTS: Although the variation in performance among anesthesiologists was statistically significant (P = 0.025), none of the anesthesiologists in the sample was classified as a high- or low-performance outliers. The contribution of anesthesiologists to outcomes represented 0.51% of the overall variability in patient outcomes (intraclass correlation coefficient [ICC] = 0.0051; 95% confidence interval [CI], 0.002-0.014), whereas the contribution of hospitals to patient outcomes was 2.90% (ICC = 0.029; 95% CI, 0.017-0.050). The anesthesiologist median odds ratio (MOR) was 1.13 (95% CI, 1.08-1.24), suggesting that the variation between anesthesiologist was modest, whereas the hospital MOR was 1.35 (95% CI, 1.25-1.48). In a separate analysis, the contribution of surgeons to overall outcomes represented 1.76% of the overall variability in patient outcomes (ICC = 0.018, 95% CI, 0.010-0.031), and the surgeon MOR was 1.26 (95% CI, 1.19-1.37). Twelve of the surgeons were identified as performance outliers. CONCLUSIONS: The impact of anesthesiologists on the total variability in cardiac surgical outcomes was probably about one-fourth as large as the surgeons' contribution. None of the anesthesiologists caring for cardiac surgical patients in New York State over a 3+ year period were identified as performance outliers. The use of a performance metric based on death or major complications for P4P may not be feasible for cardiac anesthesiologists.

The Impact of the Internet on Health Consultation Market Concentration: An Econometric Analysis of Secondary Data.

BACKGROUND: Many markets have traditionally been dominated by a few best-selling products, and this is also the case for the health care industry. However, we do not know whether the market will be more or less concentrated when health care services are delivered online (known as E-consultation), nor do we know how to reduce the concentration of the E-consultation market. OBJECTIVE: The aim of this study was to investigate the concentration of the E-consultation market and how to reduce its concentration through information disclosure mechanisms (online reputation and self-representation). METHODS: We employed a secondary data econometric analysis using transaction data obtained from an E-consultation Website (haodf.com) for three diseases (infantile pneumonia, diabetes, and pancreatic cancer) from 2008 to 2015. We included 2439 doctors in the analysis. RESULTS: The E-consultation market largely follows the 20/80 principle, namely that approximately 80% of orders are fulfilled by nearly 20% of doctors. This is much higher than the offline health care market. Meanwhile, the market served by doctors with strong online reputations (beta=0.207, P<.001) or strong online self-representation (beta=0.386, P<.001) is less concentrated. CONCLUSIONS: When health care services are delivered online, the market will be more concentrated (known as the "Superstar" effect), indicating poor service efficiency for society as a whole. To reduce market concentration, E-consultation websites should provide important design elements such as ratings of doctors (user feedback), articles contributed by doctors, and free consultation services (online representation). A possible and important way to reduce the market concentration of the E-consultation market is to accumulate enough highly rated or highly self-represented doctors.

Cost of Operating Central Cancer Registries and Factors That Affect Cost: Findings From an Economic Evaluation of Centers for Disease Control and Prevention National Program of Cancer Registries.

CONTEXT: The Centers for Disease Control and Prevention (CDC) evaluated the economics of the National Program of Cancer Registries to provide the CDC, the registries, and policy makers with the economics evidence-base to make optimal decisions about resource allocation. Cancer registry budgets are under increasing threat, and, therefore, systematic assessment of the cost will identify approaches to improve the efficiencies of this vital data collection operation and also justify the funding required to sustain registry operations. OBJECTIVES: To estimate the cost of cancer registry operations and to assess the factors affecting the cost per case reported by National Program of Cancer Registries-funded central cancer registries. METHODS: We developed a Web-based cost assessment tool to collect 3 years of data (2009-2011) from each National Program of Cancer Registries-funded registry for all actual expenditures for registry activities (including those funded by other sources) and factors affecting registry operations. We used a random-effects regression model to estimate the impact of various factors on cost per cancer case reported. RESULTS: The cost of reporting a cancer case varied across the registries. Central cancer registries that receive high-quality data from reporting sources (as measured by the percentage of records passing automatic edits) and electronic data submissions, and those that collect and report on a large volume of cases had significantly lower cost per case. The volume of cases reported had a large effect, with low-volume registries experiencing much higher cost per case than medium- or high-volume registries. CONCLUSIONS: Our results suggest that registries operate with substantial fixed or semivariable costs. Therefore, sharing fixed costs among low-volume contiguous state registries, whenever possible, and centralization of certain processes can result in economies of scale. Approaches to improve quality of data submitted and increasing electronic reporting can also reduce cost.

Public Health Practice of Population-Based Birth Defects Surveillance Programs in the United States.

CONTEXT: Birth defects remain a leading cause of infant mortality in the United States and contribute substantially to health care costs and lifelong disabilities. State population-based surveillance systems have been established to monitor birth defects, yet no recent systematic examination of their efforts in the United States has been conducted. OBJECTIVE: To understand the current population-based birth defects surveillance practices in the United States. DESIGN: The National Birth Defects Prevention Network conducted a survey of US population-based birth defects activities that included questions about operational status, case ascertainment methodology, program infrastructure, data collection and utilization, as well as priorities and challenges for surveillance programs. Birth defects contacts in the United States, including District of Columbia and Puerto Rico, received the survey via e-mail; follow-up reminders via e-mails and telephone were used to ensure a 100% response rate. RESULTS: Forty-three states perform population-based surveillance for birth defects, covering approximately 80% of the live births in the United States. Seventeen primarily use an active case-finding approach and 26 use a passive case-finding approach. These programs all monitor major structural malformations; however, passive case-finding programs more often monitor a broader list of conditions, including developmental conditions and newborn screening conditions. Active case-finding programs more often use clinical reviewers, cover broader pregnancy outcomes, and collect more extensive information, such as family history. More than half of the programs (24 of 43) reported an ability to conduct follow-up studies of children with birth defects. CONCLUSIONS: The breadth and depth of information collected at a population level by birth defects surveillance programs in the United States serve as an important data source to guide public health action. Collaborative efforts at the state and national levels can help harmonize data collection and increase utility of birth defects programs.