Assessment of quality of data submitted for NICE technology appraisals over two decades.

BACKGROUND: The National Institute for Health and Care Excellence (NICE) pioneered the Health Technology Assessment (HTA) processes and methodologies. Technology appraisals (TAs) focus on pharmaceutical products and clinical and economic data, which are presented by the product manufacturers to the NICE appraisal committee for decision-making. Uncertainty in data reduces the chance of a positive outcome from the HTA process or requires a higher discount. OBJECTIVE: To investigate the quality of clinical data (comparator, quality of life (QoL), randomised controlled trials (RCTs) and overall quality of evidence) submitted by the manufacturers to NICE. DESIGN: This retrospective evaluation analysed active TAs published between 2000 and 2019 (up to TA600). METHODS: For all TAs, we extracted data from the Assessment Group and Evidence Review Group reports and Final Appraisal Determinations on (1) the quality of submitted RCTs and (2) the overall quality of evidence submitted for decision-making. For single TAs, we also extracted data and its critique on QoL and comparators. Each category was scored for quality and analysed using descriptive statistics. RESULTS: 409 TAs were analysed (multiple technology appraisals (MTA)=104, single technology appraisal (STA)=305). In two-thirds of TAs, the overall quality of evidence was either poor (n=224, 55%) or unacceptable (n=41, 10%). In 39% (n=119) of the STAs, the quality of comparative evidence was considered poor, and in 17% (n=51) unacceptable. In 44% (n=135) of STAs, the quality of QoL data was considered poor, 15% (n=47) unacceptable, 33% (n=102) acceptable and 7% (n=21) as good. Over 20 years of longitudinal analysis did not show improvements in the quality of evidence submitted to NICE. CONCLUSION: We found that the primary components of clinical evidence influencing NICE's decision-making framework were of poor quality. It is essential to continue to generate robust clinical data for premarket and postmarket introduction of medicines into clinical practice to ensure they deliver benefits to patients.

The Origin of First-in-Class Drugs: Innovation Versus Clinical Benefit.

First-in-class (FIC) designation became a hallmark of innovation, however, even at the marketing authorization stage, little is known about the clinical benefits these products deliver. We identified the provenance of the FIC drugs that entered the French market from 2008 to 2018 and matched these medicines to the clinical benefit grading by Haute Autorité de Santé (HAS) and Prescrire. Analyses were performed using descriptive statistics to present our findings by drug origin and therapeutic area and to establish the degree of concordance between HAS and Prescrire. Of the 135 FIC drugs identified, 71.1% (n = 96) originated from the industry, 16.3% (n = 22) from academia, and 12.6% (n = 17) from joint partnerships. Three therapeutic areas accounted for most FIC medications: antineoplastic (25.9%, N = 35), anti-infective (14.1%, N = 19), and metabolic (11.1%, N = 15) agents. HAS and Prescrire agreed on 60.74% of clinical benefit gradings. According to HAS, only 5% of all FIC drugs had substantial added benefit, and only 3%, according to Prescrire. HAS and Prescrire graded 45.9% and 68.2%, respectively, of FIC drugs as no clinical benefit and 48.9% and 28.9%, respectively, as some clinical benefit. FIC-designated drugs are primarily of industry (> 70%) rather than academic origin. We found that 55% of FIC medicines that entered the French market over the 10-year period deliver no additional clinical benefit. Whereas FIC medicines may represent important scientific advancements in drug development, in > 50% of cases, the new mode of action does not translate into additional clinical benefits for patients.

Provenance and Clinical Benefit of Medicines Introduced to the French Market, 2008 to 2018.

Importance: Both the commercial sector and academia play a vital role in medicine development. Ongoing debates exist on their contribution and the value of medicinal products entering the market. Objective: To identify the provenance and clinical benefit of medicines that entered the French market between 2008 and 2018. Design and Setting: In this cross-sectional study, the provenance of each medicine in the French market was established via a review of multiple sources documenting at least 2 matching findings per product. The clinical benefit was assigned using the matched scale developed from the Prescrire and Haute Autorité de Santé (HAS) gradings. The χ2 test was used to analyze the proportions and frequencies of medicines graded by Prescrire and HAS by origin, therapeutic category, and clinical benefit. Main outcomes and measures: The origins and therapeutic categories of medicines. Clinical benefit based on Prescrire and HAS grading. Concordance of Prescrire and HAS grading. Results: Of the 632 medicines that entered the French market between 2008 and 2018, 464 originated (73%) in the commercial sector, and 168 originated (27%) in the academic setting or in collaboration with commercial enterprises. Prescrire graded psychotropic agents (13/14 [93%]), whereas HAS graded respiratory agents (24/25 [96%]) as the highest percentage of medicines that provided no added benefit. Prescrire graded 360 medicines (77.6%) that originated in the industry and 108 medicines (64.3%) that originated in the academic setting (P = .001) to have no added clinical benefit. HAS assigned such grading to 331 ([71.3%] industry) vs 104 ([61.9%] academia) (P = .02). Based on the Prescrire grading, academia invented more medicines delivering some added benefit 57 (33.9%) vs 98 (21.1%) invented by industry (P = .001). HAS grading on some added benefit 51 ([30.4%] academia) vs 121 ([26.1%] industry) did not reach statistical significance (P = .29). However, HAS grading on substantial added clinical benefit reached statistical significance in favor of academia (13 [7.7%] vs 12 [2.6%] in the industry; P = .003), whereas Prescrire grading did not (1.8% academia vs 1.3% industry; P = .64). Conclusions and Relevance: More than 70% of medicines that entered the French market during the 10-year period originated in the commercial sector. Although most medicines were not graded as providing clinical benefit, medicines originating in the academic setting were more likely to be graded as conferring clinical benefit than those originating in the commercial setting.

Costs and Causes of Oncology Drug Attrition With the Example of Insulin-Like Growth Factor-1 Receptor Inhibitors.

Importance: The development of oncology drugs is expensive and beset by a high attrition rate. Analysis of the costs and causes of translational failure may help to reduce attrition and permit the more appropriate use of resources to reduce mortality from cancer. Objective: To analyze the causes of failure and expenses incurred in clinical trials of novel oncology drugs, with the example of insulin-like growth factor-1 receptor (IGF-1R) inhibitors, none of which was approved for use in oncology practice. Design, Setting, and Participants: In this cross-sectional study, inhibitors of the IGF-1R and their clinical trials for use in oncology practice between January 1, 2000, and July 31, 2021, were identified by searching PubMed and ClinicalTrials.gov. A proprietary commercial database was interrogated to provide expenses incurred in these trials. If data were not available, estimates were made of expenses using mean values from the proprietary database. A search revealed studies of the effects of IGF-1R inhibitors in preclinical in vivo assays, permitting calculation of the percentage of tumor growth inhibition. Archival data on the clinical trials of IGF-1R inhibitors and proprietary estimates of their expenses were examined, together with an analysis of preclinical data on IGF-1R inhibitors obtained from the published literature. Main Outcomes and Measures: Expenses associated with research and development of IGF-1R inhibitors. Results: Sixteen inhibitors of IGF-1R studied in 183 clinical trials were found. None of the trials, in a wide range of tumor types, showed efficacy permitting drug approval. More than 12 000 patients entered trials of IGF-1R inhibitors in oncology indications in 2003 to 2021. These trials incurred aggregate research and development expenses estimated at between $1.6 billion and $2.3 billion. Analysis of the results of preclinical in vivo assays of IGF-1R inhibitors that supported subsequent clinical investigations showed mixed activity and protocols that poorly reflected the treatment of advanced metastatic tumors in humans. Conclusions and Relevance: Failed drug development in oncology incurs substantial expense. At an industry level, an estimated $50 billion to $60 billion is spent annually on failed oncology trials. Improved target validation and more appropriate preclinical models are required to reduce attrition, with more attention to decision-making before launching clinical trials. A more appropriate use of resources may better reduce cancer mortality.

The early impact of the global lockdown on post-secondary students and staff: A global, descriptive study.

OBJECTIVES: The aim of this study was to gain a preliminary, broad-level understanding of how the first lockdown impacted post-secondary students, faculty, and staff worldwide. METHODS: The data were obtained via a global online cross-sectional questionnaire survey using a mixed-method design and disseminated to university students, faculty, and staff from April to November 2020. The data were categorized in four themes/categories: (1) social life and relationships, (2) access to services, (3) health experiences, and (4) impact on mental health well-being. RESULTS: The survey included 27,804 participants from 121 countries and 6 continents. The majority of participants were from Europe (73.6%), female (59.2%), under 30 years of age (64.0%), living in large urban areas (61.3%), %), and from middle-income families (66.7%). Approximately 28.4% of respondents reported that the lockdown negatively impacted their social life, while 21.2% reported the lockdown had a positive impact. A total of 39.2% reported having issues accessing products or services, including essentials, such as groceries, or medical services. In addition, respondents reported an increase in stress and anxiety levels and a decrease in quality of life during the first 2 weeks of the lockdown. CONCLUSIONS: The COVID-19 pandemic and lockdown measures had an evident impact on the lives of post-secondary students, faculty, and staff. Further research is required to inform and improve policies to support these populations at both institutional and national levels.

Audit of data redaction practices in NICE technology appraisals from 1999 to 2019.

OBJECTIVES: To assess the extent and type of data redaction in all active technology appraisals (TA) and highly specialised technology (HST) evaluations issued by the National Institute for Health and Care Excellence (NICE) from its conception of the institute to September 2019. To propose policy recommendations for transparency. METHODS: Structured audit to establish extent of data redaction-proportion of appraisals and specific data categories and assess redaction by: indication, appraisal process, manufacturer, type of data-price, adverse events (AEs), clinical (excluding AEs), incremental quality-adjusted life-years. Longitudinal analysis over 20 years. RESULTS: All TAs with available documentation and active recommendations (n=408) and HSTs (n=10) published from March 2000 to 11 September 2019 have been assessed for data redaction. Overall, 333 TAs (81.6%) have data redaction, 86 (25.8%) of them are heavily redacted. Clinical data (excluding AEs) are redacted in 268 (65.7%) appraisals, AE data in 128 (31.4%), price in 238 (58.3%). In total, 87% of oncology appraisals have redacted data vs 78% of non-oncology appraisals. 91% of single TAs have redacted data vs 59% of multiple TAs. 25% of final guidance documents (e.g. Final Appraisal Determination - FAD) do not report one or more instance of clinical data. Data redaction increased substantially over time, and is currently at its highest level with 100% of TAs having at least some data redaction in 2019/2020, 96% of appraisals in 2018/2019% and 94% of appraisals in 2017/2018. All 10 HST evaluations have redacted data, with 4 of them being heavily redacted. CONCLUSIONS: Documents supporting NICE TA and HST recommendations are significantly redacted, thereby concealing clinical and economic data of importance to patients, clinicians and researchers. Documents remain redacted on the NICE website for years. Policy change is required to ensure transparency of data underpinning NICE's decisions.

NICE's evaluations of medicines authorized by EMA with conditional marketing authorization or under exceptional circumstances.

OBJECTIVES: To investigate the impact of the uncertainty stemming from products with European conditional marketing authorization (CMA) or authorization in exceptional circumstances (AEC) on the National Institute for Health and Care Excellence's (NICE) recommendations. METHODS: Products which received CMA/AEC by European Medicines Agency (EMA) up to 1 December 2016 were identified and matched with corresponding NICE decisions issued by August 2017, the status of which was then traced to August 2019. We assessed whether the conversion of CMA to full marketing authorization triggered a review of a NICE decision. The odds of a recommendation carrying a commercial arrangement for products with and without CMA/AEC were calculated. RESULTS: Fifty-four products were granted CMA/AEC by EMA. NICE conducted thirty evaluations of products with CMA/AEC. Twelve products were recommended by NICE by August 2017 and fourteen by August 2019. All recommendations had an associated commercial arrangement. The odds of carrying a commercial arrangement were higher for products with CMA/AEC compared to those with full authorization. Conversions from conditional to full authorization among products not recommended by NICE did not trigger an appraisal review. CONCLUSIONS: Uncertainty, stemming from the lack of robust clinical data of products authorized with CMA/AEC, has a substantial impact on HTA recommendations, frequently requiring risk mitigation mechanisms such as commercial and data collection arrangements. Further analyses should be conducted to assess whether the benefits of early access strategies outweigh the risks for patients and the healthcare system.

Assessment of technical errors and validation processes in economic models submitted by the company for NICE technology appraisals.

BACKGROUND: Economic models play a central role in the decision-making process of the National Institute for Health and Care Excellence (NICE). Inadequate validation methods allow for errors to be included in economic models. These errors may alter the final recommendations and have a significant impact on outcomes for stakeholders. OBJECTIVE: To describe the patterns of technical errors found in NICE submissions and to provide an insight into the validation exercises carried out by the companies prior to submission. METHODS: All forty-one single technology appraisals (STAs) completed in 2017 by NICE were reviewed and all were on medicines. The frequency of errors and information on their type, magnitude, and impact was extracted from publicly available NICE documentation along with the details of model validation methods used. RESULTS: Two STAs (5 percent) had no reported errors, nineteen (46 percent) had between one and four errors, sixteen (39 percent) had between five and nine errors, and four (10 percent) had more than ten errors. The most common errors were transcription errors (29 percent), logic errors (29 percent), and computational errors (25 percent). All STAs went through at least one type of validation. Moreover, errors that were notable enough were reported in the final appraisal document (FAD) in eight (20 percent) of the STAs assessed but each of these eight STAs received positive recommendations. CONCLUSIONS: Technical errors are common in the economic models submitted to NICE. Some errors were considered important enough to be reported in the FAD. Improvements are needed in the model development process to ensure technical errors are kept to a minimum.

Young people's preferences for the use of emerging technologies for asymptomatic regular chlamydia testing and management: a discrete choice experiment in England.

OBJECTIVE: To undertake a comprehensive assessment of the strength of preferences among young people for attributes of emerging technologies for testing and treatment of asymptomatic chlamydia. DESIGN: Discrete choice experiment (DCE) with sequential mixed methods design. A staged approach to selection of attributes/levels included two literature reviews, focus groups with young people aged 16-24 years (n=21), experts' review (n=13) and narrative synthesis. Cognitive testing was undertaken to pilot and adapt the initial questionnaire. Online national panel was used for final DCE survey to maximise generalisability. Analysis of questionnaire responses used multinomial logit models and included validity checks. SETTING: England. PARTICIPANTS: 1230 young people aged 16-24 from a national online panel (completion rate 73%). OUTCOME MEASURES: ORs for service attributes in relation to reference levels. RESULTS: The strongest attribute influencing preferences was chlamydia test accuracy (OR 3.24, 95% CI 3.13 to 3.36), followed by time to result (OR 1.81, 95% CI 1.71 to 1.91). Respondents showed a preference for remote chlamydia testing options (self-testing, self-sampling and postal testing) over attendance at a testing location. For accessing treatment following a positive test result, there was a general preference for online (OR 1.21, 95% CI 1.15 to 1.28) versus traditional general practitioner (OR 1.18, 95% CI 1.12 to 1.24) or pharmacy (OR 1.15, 95% CI 1.10 to 1.22) over clinic services. For accessing a healthcare professional and receipt of antibiotics, there was little difference in preferences between options. CONCLUSIONS: Both test accuracy and very short intervals between testing and results were important factors for young people when deciding whether to undergo a routine test for asymptomatic chlamydia, with test accuracy being more important. These findings should assist technology developers, policymakers, commissioners and service providers to optimise technology adoption in service redesign, although use of an online panel may limit generalisability of findings to other populations.

A review of NICE appraisals of pharmaceuticals 2000-2016 found variation in establishing comparative clinical effectiveness.

OBJECTIVE: To identify and assess the methods for estimating comparative clinical effectiveness for novel pharmaceutical products licensed on the basis of nonrandomized controlled trial (non-RCT) data and to evaluate the corresponding National Institute for Health and Care Excellence (NICE) recommendations. METHODS: Our identification strategy was twofold. First, we reviewed all NICE appraisals between 2010 and 2016 and identified technologies where comparative clinical effectiveness estimates were calculated using non-RCT data. Second, we checked if NICE appraisals completed from 2000 to 2010 had included pharmaceuticals that were granted European Medicines Agency marketing authorization without RCT data between 1999 and 2014. Information was extracted on the method used to establish comparative clinical effectiveness as well as the corresponding NICE recommendations. We also collected information on the rationale for utilizing non-RCT data in NICE appraisals. RESULTS: Of 489 individual pharmaceutical technologies assessed by NICE, 22 (4%) used non-RCT data to estimate comparative clinical effectiveness. Methods for establishing external controls in such studies varied: 13 (59%) used published trials, 6 (27%) used observational data, 2 (9%) used expert opinion, and 1 (5%) used a responder vs nonresponder analysis. Only 5 (23%) used a regression model to adjust for covariates. We did not observe a notable difference in the proportion of pharmaceutical technologies that received a positive recommendation from NICE whether the decision was based on RCT or non-RCT data (83% vs 86%). CONCLUSIONS: To date, a small number of appraisals by NICE based on non-RCT data did not result in substantially different treatment decisions. The majority of the technologies appraised on the basis of non-RCT data either received a positive recommendation or a positive recommendation with restrictions. The methods used to calculate comparative clinical effectiveness estimates varied, highlighting the need to establish clear guidance.

Assessment of Devices, Diagnostics and Digital Technologies: A Review of NICE Medical Technologies Guidance.

BACKGROUND: The Medical Technologies Evaluation Programme (MTEP) of NICE in England aims to evaluate medical devices that are deemed to be cost-saving or cost-neutral and produce Medical Technology Guidance (MTG) to encourage their adoption. OBJECTIVE: To review the MTGs since MTEP's inception in 2009 until February 2017. METHODS: One researcher assessed all published MTGs and extracted data on the clinical and economic evidence supporting each technology. The NICE Committee's decision outcome for each assessment was also recorded. A qualitative analysis was performed on technologies that were not supported for adoption to identify the main drivers of the decision. RESULTS: Thirty-one MTGs were reviewed. The committee fully supported the medical devices in 14 MTGs, 11 were partially supported and six were not supported. Of the MTGs, 58% had no RCT data available and the main source of evidence came from non-experimental studies. There was no statistically significant difference in the average number of RCTs and non-experimental studies between the fully-supported, partially-supported, and not-supported technologies. Whilst all the fully-supported MTGs demonstrated cost-saving results, only 50% of the not-supported MTGs did. The sponsor estimated a higher average cost-saving than the EAC in most of the cases (20/31). The qualitative evaluation suggests that the main drivers for negative decisions were the quantity or quality of studies, and costs incurred in the economic evaluation results. CONCLUSIONS: The main drivers of the decision-making process are the quality and quantity of the submitted evidence supporting the technologies, as well as the economic evaluation results.

The impact of parallel regulatory-health technology assessment scientific advice on clinical development. Assessing the uptake of regulatory and health technology assessment recommendations.

AIMS: The parallel regulatory-health technology assessment scientific advice (PSA) procedure allows manufacturers to receive simultaneous feedback from both EU regulators and health technology assessment (HTA) bodies on development plans for new medicines. The primary objective of the present study is to investigate whether PSA is integrated in the clinical development programmes for which advice was sought. METHODS: Contents of PSA provided by regulators and HTA bodies for each procedure between 2010 and 2015 were analysed. The development of all clinical studies for which PSA had been sought was tracked using three different databases. The rate of uptake of the advice provided by regulators and HTA bodies was assessed on two key variables: comparator/s and primary endpoint. RESULTS: In terms of uptake of comparator recommendations at the time of PSA in the actual development, our analysis showed that manufacturers implemented comparators to address both the needs of regulators and of at least one HTA body in 12 of 21 studies. For primary endpoints, in all included studies manufacturers addressed both the needs of the regulators and at least one HTA body. CONCLUSIONS: One of the key findings of this analysis is that manufacturers tend to implement changes to the development programme based on both regulatory and HTA advice with regards to the choice of primary endpoint and comparator. It also confirms the challenging choice of the study comparator, for which manufacturers seem to be more inclined to satisfy the regulatory advice. Continuous research efforts in this area are of paramount importance from a public health perspective.

Integrating health technology assessment requirements in the clinical development of medicines: the experience from NICE scientific advice.

PURPOSE: The primary objective of the study was to analyse the proposed clinical development and economic evaluation plans for investigational medicinal products for which pharmaceutical companies have sought health technology assessment (HTA) scientific advice (SA). METHODS: We have selected and analysed all the scientific advice procedures undertaken by National Institute for Health and Care Excellence (NICE) SA between 1 January 2009 and 3 December 2015 for investigational medicinal products. We have mapped the questions asked by the companies and the areas of advice highlighted in the advice reports to the sections of the NICE methods guide to the technology appraisals (2013). RESULTS: An overwhelming proportion of SA procedures have addressed questions related to the clinical development and specifically the main pivotal efficacy studies. Approximately a quarter of the questions relate to the approaches to economic evaluation. Questions raised in European Medicines Agency-HTA procedures generally focus on clinical efficacy issues whereas cost-effectiveness ones tend to dominate in NICE-only procedures. Our analysis shows that the issues mostly discussed in the HTA SA are the choice of comparator, the generalisability of the clinical trial evidence to the NHS practice and the impact of the clinical trial outcomes on quality of life and survival. Less disagreement with the developers' plans was seen in the choice of clinical endpoints, population definition, position of the technology in the treatment pathway and study design. CONCLUSIONS: Scientific advice is designed to improve the quality of evidence and approaches to evidence generation for future regulatory approval and HTA evaluation. Our experience to date suggests that payer requirements are inconsistently integrated in the clinical development programmes. More efforts should be dedicated to demonstrating the clinical value of new medicinal products to patients and key decision-makers.

How aligned are the perspectives of EU regulators and HTA bodies? A comparative analysis of regulatory-HTA parallel scientific advice.

BACKGROUND: In 2010, the European Medicines Agency (EMA) initiated a pilot project on parallel scientific advice with Health Technology Assessment bodies (HTABs) that allows manufacturers to receive simultaneous feedback from both the European Union (EU) regulators and HTABs on their development plans for medicines. AIMS: The present retrospective qualitative analysis aimed to explore how the parallel scientific advice system is working and levels of commonality between the EU regulators and HTABs, and among HTABs, when applicants obtain parallel scientific advice from both a regulatory and an HTA perspective. METHODS: We analysed the minutes of discussion meetings held at the EMA between 2010, when parallel advice was launched, and 1 May 2015, when the cutoff date for data extraction was set. The analysis was based on predefined criteria and conducted at two different levels of comparison: the answers of the HTABs vs. those of the regulators, and between the answers of the participating HTA agencies. RESULTS: The analysis was based on 31 procedures of parallel scientific advice. The level of full agreements was highest for questions on patient population (77%), while disagreements reached a peak for questions on the study comparator (30%). With regard to comparisons among HTABs, there was a high level of agreement for all domains. CONCLUSIONS: There is evident commonality, in terms of evidence requirements between the EU regulators and participating HTABs, as well as among HTABs, on most aspects of clinical development. Indeed, regardless of the question content, the analysis showed that a high level of overall agreement was reached through the process of parallel scientific advice.

A new fetal RHD genotyping test: costs and benefits of mass testing to target antenatal anti-D prophylaxis in England and Wales.

BACKGROUND: Postnatal and antenatal anti-D prophylaxis have dramatically reduced maternal sensitisations and cases of rhesus disease in babies born to women with RhD negative blood group. Recent scientific advances mean that non-invasive prenatal diagnosis (NIPD), based on the presence of cell-free fetal DNA in maternal plasma, could be used to target prophylaxis on "at risk" pregnancies where the fetus is RhD positive. This paper provides the first assessment of cost-effectiveness of NIPD-targeted prophylaxis compared to current policies. METHODS: We conducted an economic analysis of NIPD implementation in England and Wales. Two scenarios were considered. Scenario 1 assumed that NIPD will be only used to target antenatal prophylaxis with serology tests continuing to direct post-delivery prophylaxis. In Scenario 2, NIPD would also displace postnatal serology testing if an RhD negative fetus was identified. Costs were estimated from the provider's perspective for both scenarios together with a threshold royalty fee per test. Incremental costs were compared with clinical implications. RESULTS: The basic cost of an NIPD in-house test is £16.25 per sample (excluding royalty fee). The two-dose antenatal prophylaxis policy recommended by NICE is estimated to cost the NHS £3.37 million each year. The estimated threshold royalty fee is £2.18 and £8.83 for Scenarios 1 and 2 respectively. At a £2.00 royalty fee, mass NIPD testing would produce no saving for Scenario 1 and £507,154 per annum for Scenario 2. Incremental cost-effectiveness analysis indicates that, at a test sensitivity of 99.7% and this royalty fee, NIPD testing in Scenario 2 will generate one additional sensitisation for every £9,190 saved. If a single-dose prophylaxis policy were implemented nationally, as recently recommended by NICE, Scenario 2 savings would fall. CONCLUSIONS: Currently, NIPD testing to target anti-D prophylaxis is unlikely to be sufficiently cost-effective to warrant its large scale introduction in England and Wales. Only minor savings are calculated and, balanced against this, the predicted increase in maternal sensitisations may be unacceptably high. Reliability of NIPD assays still needs to be demonstrated rigorously in different ethnic minority populations. First trimester testing is unlikely to alter this picture significantly although other emerging technologies may.

Non-invasive fetal RHD genotyping tests: a systematic review of the quality of reporting of diagnostic accuracy in published studies.

Articles reporting the diagnostic accuracy of non-invasive prenatal diagnostic (NIPD) tests for RHD genotyping using fetal material extracted from maternal blood have been published steadily for over a decade. Health care providers in Europe have started to use this technology for management of the small number of sensitised pregnancies (ca. 220-600 per annum in the Netherlands, Germany, France and the UK). Scientists and clinicians are also advocating widespread implementation for the far larger number of non-sensitised RhD-negative pregnancies (ca. 34,000-125,000 per annum in the same countries). Large-scale, prospective trials are only now underway. Estimates of the technical performance of these tests are currently based on results from small-scale studies, together with formal meta-analysis. The issue of early assessment of test performance is one faced by many new genetic tests. As part of a wider study we have investigated the quality of reporting of diagnostic accuracy in publications and produced guidelines for future studies. A systematic search of the literature identified 27 papers which met predefined inclusion criteria. All 27 papers were, first, assessed against an international quality (STARD) checklist for reporting of diagnostic accuracy and, second, against our own in-house NIPD proforma to assess the implications of the quality of reporting specifically for the RhD NIPD test. Authors were found to generally present an optimistic view of NIPD, bearing in mind weaknesses identified in reporting and conduct of their studies and the analysis of results, as evidenced by the low STARD scores. The NIPD proforma identified that specific biases were potentially introduced through selective population sampling and/or failure to report the make-up of the population tested, omission of inconclusive results, inconsistencies in the handling of repeat results on a sample, and lack of adequate controls. These factors would inevitably affect the validity of diagnostic accuracy as reported in individual publications, as well as any subsequent meta-analyses. Together, published reports to date may provide a biased picture of the actual potential of NIPD testing for fetal RHD genotyping. Generalisation of the available evidence on diagnostic accuracy, especially to large-scale implementation of NIPD testing of non-sensitised women, will also require that decision makers consider further aspects such as test reliability and cost of routine testing in clinical practice. It is recommended that all studies of diagnostic accuracy of NIPD tests adhere to the STARD quality checklist in order to improve reporting, thereby, minimising bias and increasing the comparability of studies. Researchers should also consider specific shortcomings for NIPD and avoid selective participant sampling; report population characteristics; report handling of replicate sampling as well as their failure rates; and include controls for genotypes tested in the study. Furthermore, meta-analyses should consider the quality, as well as the sample size, of NIPD studies in their analysis. Larger trials, required to produce results that are valid and meaningful for clinical practice, must also adhere to these reporting standards.