A Comprehensive Framework for Evaluating the Value Created by Real-World Evidence for Diverse Stakeholders: The Case for Coordinated Registry Networks.

OBJECTIVES: This manuscript presents a comprehensive framework for the assessment of the value of real-world evidence (RWE) in healthcare decision-making. While RWE has been proposed to overcome some limitations of traditional, one-off studies, no systematic framework exists to measure if RWE actually lowers the burden. This framework aims to fill that gap by providing conceptual approaches for evaluating the time and cost efficiencies of RWE, thus guiding strategic investments in RWE infrastructure. METHODS: The framework consists of four components: (114th Congress. 21st Century Cures Act.; 2015. https://www.congress.gov/114/plaws/publ255/PLAW-114publ255.pdf .) identification of stakeholders using and producing RWE, (National Health Council. Glossary of Patient Engagement Terms. Published 2019. Accessed May 18. 2021. https://nationalhealthcouncil.org/glossary-of-patient-engagement-terms/ .) understanding value propositions on how RWE can benefit stakeholders, (Center for Drug Evaluation and Research. CDER Patient-Focused Drug Development. U.S. Food & Drug Administration.) defining key performance indicators (KPIs), and (U.S. Department of Health and Human Services - Food and Drug Administration: Center for Devices and Radiological Health and Center for Biologics Evaluation and Research. Use of Real-World Evidence to Support Regulatory Decision-Making for Medical Devices - Guidance for Industry and Food and Drug Administration Staff. 2017. http://www.fda.gov/BiologicsBloodVaccines/GuidanceComplianceRegulatoryInformation/Guida .) establishing metrics and case studies to assess value. KPIs are categorized as 'better, faster, or cheaper" as an indicator of value: better focusing on high-quality actionable evidence; 'faster,' denoting time-saving in evidence generation, and 'cheaper,' emphasizing cost-efficiency decision compared to methodologies that do not involve data routinely collected in clinical practice. Metrics and relevant case studies are tailored based on stakeholder value propositions and selected KPIs that can be used to assess what value has been created by using RWE compared to traditional evidence-generation approaches and comparing different RWE sources. RESULTS: Operationalized through metrics and case studies drawn from the literature, the value of RWE is documented as improving treatment effect heterogeneity evaluation, expanding medical product labels, and expediting post-market compliance. RWE is also shown to reduce the cost and time required to produce evidence compared to traditional one-off approaches. An original example of a metric that measures the time saved by RWE methods to detect a signal of a product failure was presented based on analysis of the National Cardiovascular Disease Registry. CONCLUSIONS: The framework presented in this manuscript offers a comprehensive approach for evaluating the value of RWE, applicable to all stakeholders engaged in leveraging RWE for healthcare decision-making. Through the proposed metrics and illustrated case studies, valuable insights are provided into the heightened efficiency, cost-effectiveness, and improved decision-making within clinical and regulatory domains facilitated by RWE. While this framework is primarily focused on medical devices, it could potentially inform the determination of RWE value in other medical products. By discerning the variations in cost, time, and data utility among various evidence-generation methods, stakeholders are empowered to invest strategically in RWE infrastructure and shape future research endeavors.

Developing a Coordinated Registry Network for devices used for acute ischemic stroke intervention: basilar artery occlusion quality assessment pilot.

BACKGROUND: Real-world data can be helpful in evaluating endovascular therapy (EVT) in ischemic stroke care. We conducted a pilot study to aggregate data on basilar artery occlusion (BAO) EVT from existing registries in the USA. We evaluated the availability, completeness, quality, and consistency of common data elements (CDEs) across data sources. METHODS: We harmonized patient-level data from five registry data sources and assessed the availability, completeness (defined by the presence in at least four data sources), and consistency of CDEs. We assessed data quality based on seven pre-defined critical domains for BAO EVT investigation: baseline patient and disease characteristics; time metrics; description of intervention; adjunctive devices, revascularization scores, complications; post-intervention National Institutes of Health Stroke Scale scores; discharge disposition; 30-day and 90-day mortality and modified Rankin Scale (mRS) scores. RESULTS: The aggregated dataset of five registries included 493 BAO procedures between January 2013 and January 2020. In total, 88 CDEs were screened and 35 (40%) elements were considered prevalent. Of these 35 CDEs, the majority were collected for >80% of cases when aggregated. All seven pre-defined domains for BAO device investigation could be fulfilled with harmonized data elements. Most data elements were collected with consistent or compatible definitions across registries. The main challenge was the collection of 90-day outcomes. CONCLUSIONS: This pilot shows the feasibility of aggregating and harmonizing critical CDEs across registries to create a Coordinated Registry Network (CRN). The CRN with partnerships between multiple registries and stakeholders could help improve the breadth and/or depth of real-world data to help answer relevant questions and support clinical and regulatory decisions.

Validation of algorithms in studies based on routinely collected health data: general principles.

Clinicians, researchers, regulators, and other decision-makers increasingly rely on evidence from real-world data (RWD), including data routinely accumulating in health and administrative databases. RWD studies often rely on algorithms to operationalize variable definitions. An algorithm is a combination of codes or concepts used to identify persons with a specific health condition or characteristic. Establishing the validity of algorithms is a prerequisite for generating valid study findings that can ultimately inform evidence-based health care. This paper aims to systematize terminology, methods, and practical considerations relevant to the conduct of validation studies of RWD-based algorithms. We discuss measures of algorithm accuracy; gold/reference standard; study size; prioritizing accuracy measures; algorithm portability; and implication for interpretation. Information bias is common in epidemiologic studies, underscoring the importance of transparency in decisions regarding choice and prioritizing measures of algorithm validity. The validity of an algorithm should be judged in the context of a data source, and one size does not fit all. Prioritizing validity measures within a given data source depends on the role of a given variable in the analysis (eligibility criterion, exposure, outcome or covariate). Validation work should be part of routine maintenance of RWD sources.

Venous Access: National Guideline and Registry Development (VANGUARD): Advancing Patient-Centered Venous Access Care Through the Development of a National Coordinated Registry Network.

There are over 8 million central venous access devices inserted each year, many in patients with chronic conditions who rely on central access for life-preserving therapies. Central venous access device-related complications can be life-threatening and add tens of billions of dollars to health care costs, while their incidence is most likely grossly mis- or underreported by medical institutions. In this communication, we review the challenges that impair retention, exchange, and analysis of data necessary for a meaningful understanding of critical events and outcomes in this clinical domain. The difficulty is not only with data extraction and harmonization from electronic health records, national surveillance systems, or other health information repositories where data might be stored. The problem is that reliable and appropriate data are not recorded, or falsely recorded, at least in part because policy, payment, penalties, proprietary concerns, and workflow burdens discourage completeness and accuracy. We provide a roadmap for the development of health care information systems and infrastructure that address these challenges, framed within the context of research studies that build a framework of standardized terminology, decision support, data capture, and information exchange necessary for the task. This roadmap is embedded in a broader Coordinated Registry Network Learning Community, and facilitated by the Medical Device Epidemiology Network, a Public-Private Partnership sponsored by the US Food and Drug Administration, with the scope of advancing methods, national and international infrastructure, and partnerships needed for the evaluation of medical devices throughout their total life cycle.

The association of major adverse limb events and combination stent and atherectomy in patients undergoing revascularization for lower extremity peripheral artery disease.

BACKGROUND: The effectiveness of combined atherectomy and stenting relative to use of each procedure alone for the treatment of lower extremity peripheral artery disease has not been evaluated. AIMS: The objective of this study was to evaluate the short- and long-term major adverse limb event (MALE) following the receipt of stenting, atherectomy, and the combination of stent and atherectomy. METHODS: A retrospective cohort of patients undergoing atherectomy, stent, and combination stent atherectomy for lower extremity peripheral artery disease was derived from the Vascular Quality Initiative (VQI) data set. The primary outcome was MALE and was assessed in the short-term and long-term. Short-term MALE was assessed immediately following the procedure to discharge and estimated using logistic regression. Long-term MALE was assessed after discharge to end of follow-up and estimated using the Fine-Gray subdistribution hazard model. RESULTS: Among the 46,108 included patients, 6896 (14.95%) underwent atherectomy alone, 35,774 (77.59%) received a stent, and 3438 (7.5%) underwent a combination of stenting and atherectomy. The adjusted model indicated a significantly higher odds of short-term MALE in the atherectomy group (OR = 1.35; 95% confidence interval [CI]:1.16-1.57), and not significantly different odds (OR = 0.93; 95% CI:0.77-1.13) in the combination stent and atherectomy group when compared to stenting alone. With regard to long-term MALE, the model indicated that the likelihood of experiencing the outcome was slightly lower (HR = 0.90; 95% CI:0.82-0.98) in the atherectomy group, and not significantly different (HR = 0.92; 95% CI:0.82-1.04) in the combination stent and atherectomy group when compared to the stent group. CONCLUSIONS: Patients in the VQI data set who received combination stenting and atherectomy did not experience significantly different rates of MALE when compared with stenting alone. It is crucial to consider and further evaluate the influence of anatomical characteristics on treatment strategies and potential differential effects of comorbidities and other demographic factors on the short and long-term MALE risks.

Integrating Patient-Generated Health Data Throughout the Total Product Life Cycle of Medical Devices.

Nearly ubiquitous use of personal electronics, wearable sensors, and other types of digital health technologies, along with wireless connectivity, makes the capture of health data directly from an individual easier, enabling the use of patient-generated health data (PGHD) as a potential bridge between a patient's home and the healthcare system. This type of real-world data may be a completely new type of information, or it may be a more frequent collection of traditional information over longer time periods to form a longitudinal view of a patient's health status that can inform decision-making in clinical, medical product regulatory, and coverage and reimbursement settings. The U.S. Food and Drug Administration's Center for Devices and Radiological Health (CDRH) has been exploring and advancing the collection and use of PGHD since 2016, hosting a public meeting on the topic in May 2021. This manuscript presents highlights from various discussions at this meeting including those on the importance of stakeholder engagement, characteristics of high data quality, and PGHD in practice in patient-driven registries, as well as a look forward to some of the opportunities in the field.

Building the foundation for a modern patient-partnered infrastructure to study temporomandibular disorders.

Background: Conflicting reports from varying stakeholders related to prognosis and outcomes following placement of temporomandibular joint (TMJ) implants gave rise to the development of the TMJ Patient-Led RoundTable initiative. Following an assessment of the current availability of data, the RoundTable concluded that a strategically Coordinated Registry Network (CRN) is needed to collect and generate accessible data on temporomandibular disorder (TMD) and its care. The aim of this study was therefore to advance the clinical understanding, usage, and adoption of a core minimum dataset for TMD patients as the first foundational step toward building the CRN. Methods: Candidate data elements were extracted from existing data sources and included in a Delphi survey administered to 92 participants. Data elements receiving less than 75% consensus were dropped. A purposive multi-stakeholder sub-group triangulated the items across patient and clinician-based experience to remove redundancies or duplicate items and reduce the response burden for both patients and clinicians. To reliably collect the identified data elements, the identified core minimum data elements were defined in the context of technical implementation within High-performance Integrated Virtual Environment (HIVE) web-application framework. HIVE was integrated with CHIOS™, an innovative permissioned blockchain platform, to strengthen the provenance of data captured in the registry and drive metadata to record all registry transaction and create a robust consent network. Results: A total of 59 multi-stakeholder participants responded to the Delphi survey. The completion of the Delphi surveys followed by the application of the required group consensus threshold resulted in the selection of 397 data elements (254 for patient-generated data elements and 143 for clinician generated data elements). The infrastructure development and integration of HIVE and CHIOS™ was completed showing the maintenance of all data transaction information in blockchain, flexible recording of patient consent, data cataloging, and consent validation through smart contracts. Conclusion: The identified data elements and development of the technological platform establishes a data infrastructure that facilitates the standardization and harmonization of data as well as perform high performance analytics needed to fully leverage the captured patient-generated data, clinical evidence, and other healthcare ecosystem data within the TMJ/TMD-CRN.

A Quantitative Framework to Identify and Prioritize Opportunities in Biomedical Product Innovation: A Proof-of-Concept Study.

Importance: Prioritization and funding for health initiatives, including biomedical innovation, may not consistently target unmet public health needs. Objective: To (1) develop a quantitative, databased framework to identify and prioritize opportunities for biomedical product innovation investments based on a multicriteria decision-making model (MCDM) that includes comprehensive measures of public health burden and health care costs, and (2) pilot test the model. Design, Setting, and Participants: The Department of Health and Human Services (HHS) convened public and private experts to develop a model, select measures, and complete a longitudinal pilot study to identify and prioritize opportunities for investment in biomedical product innovations that have the greatest public health benefit. Cross-sectional and longitudinal data (2012-2019) for 13 pilot medical disorders were obtained from the Institute for Health Metrics Global Burden of Disease database (IHME GBD) and the National Center for Health Statistics (NCHS). Main Outcome Measures: The main outcome measure was an overall gap score reflecting high public health burden (composite measure of mortality, prevalence, years lived with disability, and health disparities), or high health care costs (composite measure of total, public, and out-of-pocket health spending) relative to low biomedical innovation. Sixteen innovation metrics were selected to reflect the pipeline of biomedical products from research and development to market approval. A higher score indicates a greater gap. Normalized composite scores were calculated for public health burden, cost, and innovation investment using the MCDM Technique for Order of Preference by Similarity to Ideal Solution method. Results: Among the 13 conditions tested in the pilot study, diabetes (0.61), osteoarthritis (0.46), and drug-use disorders (0.39) had the highest overall gap score reflecting high public health burden, or high health care costs relative to low biomedical innovation in these medical disorders. Chronic kidney disease (0.05), chronic obstructive pulmonary disease (0.09), and cirrhosis and other liver diseases (0.10) had the least amount of biomedical product innovation despite similar public health burden and health care cost scores. Conclusions: In this cross-sectional pilot study, we developed and implemented a data-driven, proof-of-concept model that can help identify, quantify, and prioritize opportunities for biomedical product innovation. Quantifying the relative alignment between biomedical product innovation, public health burden, and health care cost may help identify and prioritize investments that can have the greatest public health benefit.

Evidence-based objective performance criteria for the evaluation of hip and knee replacement devices and technologies.

BACKGROUND: Objective performance criteria (OPC) is a novel method to provide minimum performance standards and improve the regulated introduction of original or incremental device innovations in order to prevent patients from being exposed to potentially inferior designs whilst allowing timely access to improvements. We developed 2-year safety and effectiveness OPC for total hip and knee replacement (THR and TKR). METHODS: Analyses of large databases were conducted using various data sources: a systematic literature review; a direct data analysis from The Functional Outcomes Research for Comparative Effectiveness in Total Joint Replacement and Quality Improvement Registry (FORCE-TJR) and the Kaiser Permanente Implant Registry (KPIR); and claims data analyses from longitudinal discharge data in New York and California states. The literature review included U.S. patients (≥18 years) who received THR or TKR for primary end-stage osteoarthritis and prospectively collected data on patient-reported outcome measures (PROMs) from at least 100 subjects and/or 2-year implant survival for at least 250 implants. Random effects models were used for meta-analysis. RESULTS: Data were available from a total of 951 100 patients. After screening of 7979 abstracts, 294 studies underwent full-text review and 31 studies contributed to the evidence synthesis (333 995 implants). Direct data analysis of FORCE-TJR contributed 9223 joint replacement patients to the construction of OPC for effectiveness; KPIR contributed 262 044 patients for the construction of OPC for safety. Claims database analysis contributed 345 838 patients to the construction of safety OPC. OPC for safety were constructed for cumulative incidences of 2-year all-cause and septic revision (THR/TKR 2.0%/1.6% and 0.6%/0.7%), and OPC for effectiveness were constructed based on four disease-specific and three general health-related quality of life PROMs (HOOS/KOOS 87.1/80.6; HSS/KSS function 94.4/90.6; SF-12/SF-36, PCS 46.5/41.9, EQ-5D 0.88/0.84). CONCLUSION: This study is the first to construct a 2-year OPC for the safety and effectiveness of THR and TKR based on U.S. real-world data. Based on these OPC, potential benchmarks for (single-arm study) evaluation of new device innovations are suggested for a regulated and safe introduction to the (commercial) market.

Editor's Choice - Real World Study of Mortality After the Use of Paclitaxel Coated Devices in Peripheral Vascular Intervention.

OBJECTIVE: This observational cohort study examined outcomes after peripheral vascular intervention (PVI) with paclitaxel coated devices (PCD) and non-PCD, and evaluated heterogeneity of treatment effect in populations of interest. METHODS: The study included patients undergoing percutaneous transluminal angioplasty and or stent placement between 1 October 2015 and 31 December 2018 in the Vascular Quality Initiative Registry linked to Medicare claims. It determined differences in patient mortality and ipsilateral major amputation after PVI with PCD and non-PCD using Kaplan-Meier analyses and Cox regressions with inverse probability weighting in three cohorts: (A) patients treated for femoropopliteal or infrapopliteal occlusive disease with or without any other concurrent treatment (n = 11 452); (B) those treated for isolated superficial femoral or popliteal artery disease (n = 5 519); and (C) patients with inclusion criteria designed to approximate RCT populations (n = 2 278). RESULTS: The mean age of patients was 72.3 (SD = 10.9) years, and 40.6% were female. In cohort A, patients receiving PCD had a lower mortality rate (HR 0.88, 95% CI 0.79 - 0.98) than those receiving non-PCD. There was no significant difference in mortality between groups in cohort B (HR 0.91, 95% CI 0.80 - 1.04) and cohort C (HR 1.10, 95% CI 0.84 - 1.43). Patients receiving PCD did not have a significantly elevated risk of major amputation compared with those receiving non-PCD (cohort A: HR 0.84, 95% CI 0.70 - 1.00; cohort B: HR 0.84, 95% CI 0.67 - 1.06; and cohort C: HR 1.05, 95% CI 0.51 - 2.14). CONCLUSION: No increased patient mortality or major amputation was found at three years after PVI with PCD vs. non-PCD in this large, linked registry claims study, after accounting for heterogeneity of treatment effect by population. The analysis and results from three cohorts intended to mirror the cohorts of previous studies provide robust and niche real world evidence on PCD safety and help to understand and reconcile previously discrepant findings.

Data sources and applied methods for paclitaxel safety signal discernment.

Background: Following the identification of a late mortality signal, the Food and Drug Administration (FDA) convened an advisory panel that concluded that additional clinical study data are needed to comprehensively evaluate the late mortality signal observed with the use of drug-coated balloons (DCB) and drug-eluting stent (DES). The objective of this review is to (1) identify and summarize the existing clinical and cohort studies assessing paclitaxel-coated DCBs and DESs, (2) describe and determine the quality of the available data sources for the evaluation of these devices, and (3) present methodologies that can be leveraged for proper signal discernment within available data sources. Methods: Studies and data sources were identified through comprehensive searches. original research studies, clinical trials, comparative studies, multicenter studies, and observational cohort studies written in the English language and published from January 2007 to November 2021, with a follow-up longer than 36 months, were included in the review. Data quality of available data sources identified was assessed in three groupings. Moreover, accepted data-driven methodologies that may help circumvent the limitations of the extracted studies and data sources were extracted and described. Results: There were 39 studies and data sources identified. This included 19 randomized clinical trials, nine single-arm studies, eight registries, three administrative claims, and electronic health records. Methodologies focusing on the use of existing premarket clinical data, the incorporation of all contributed patient time, the use of aggregated data, approaches for individual-level data, machine learning and artificial intelligence approaches, Bayesian approaches, and the combination of various datasets were summarized. Conclusion: Despite the multitude of available studies over the course of eleven years following the first clinical trial, the FDA-convened advisory panel found them insufficient for comprehensively assessing the late-mortality signal. High-quality data sources with the capabilities of employing advanced statistical methodologies are needed to detect potential safety signals in a timely manner and allow regulatory bodies to act quickly when a safety signal is detected.

Development of a core minimum data set to advance real-world evidence generation for uterine fibroids treatment technologies.

Objectives: The accumulation of data through a prospective, multicenter coordinated registry network (CRN) is a robust and cost-effective way to gather real-world evidence on the performance of uterine fibroids treatment technologies for device-based and intervention-based studies. To develop the CRN, a group of uterine fibroids experts, consisting of representatives from professional societies, the US Food and Drug Administration, academia, industry, and the patient community, was convened to discuss the role and feasibility of the CRN and to identify the core data elements needed to assess uterine fibroid treatment technologies. Design: A Delphi method approach was employed to achieve consensus on a core minimum data set for the CRN. A series of surveys were sent to the panel and answered by each expert anonymously and individually. Results from the surveys were collected, collated, and analyzed by a study design team from Weill Cornell Medicine. Questions for the next round were based on the analysis process and discussed with group members via a conference call. This process was repeated twice over a 3-month time period until consensus was achieved. Results: Twenty-nine experts participated in the Delphi surveys, which began with an initial list of 200 data elements. The working group reached final consensus on 97 data elements capturing patient medical history, imaging data, procedure-related data, post-procedure data, and long-term follow-up data. Conclusions: The CRN successfully convened an expert panel on uterine fibroids treatment technologies and used the Delphi method to produce a consensus-based core set of data elements. These identified data elements include important outcomes related to efficacy and safety and thus, influence patient, provider, and regulatory decision-making about treatments for uterine fibroids. Finally, the core data elements provide the foundation of the infrastructure needed for the CRN that will allow for the comparative study of uterine fibroid treatment devices and technologies.

Building Blocks for the Long-acting and Permanent Contraceptives Coordinated Registry Network.

Objectives: A multistakeholder expert group under the Women's Health Technology Coordinated Registry Network (WHT-CRN) was organized to develop the foundation for national infrastructure capturing the performance of long-acting and permanent contraceptives. The group, consisting of representatives from professional societies, the US Food and Drug Administration, academia, industry and the patient community, was assembled to discuss the role and feasibility of the CRN and to identify the core data elements needed to assess contraceptive medical product technologies. Design: We applied a Delphi survey method approach to achieve consensus on a core minimum data set for the future CRN. A series of surveys were sent to the panel and answered by each expert anonymously and individually. Results from the surveys were collected, collated and analyzed by a study design team from Weill Cornell Medicine. After the first survey, questions for subsequent surveys were based on the analysis process and conference call discussions with group members. This process was repeated two times over a 6-month time period until consensus was achieved. Results: Twenty-three experts participated in the Delphi process. Participation rates in the first and second round of the Delphi survey were 83% and 100%, respectively. The working group reached final consensus on 121 core data elements capturing reproductive/gynecological history, surgical history, general medical history, encounter information, long-acting/permanent contraceptive index procedures and follow-up, procedures performed in conjunction with the index procedure, product removal, medications, complications related to the long-acting and/or permanent contraceptive procedure, pregnancy and evaluation of safety and effectiveness outcomes. Conclusions: The WHT-CRN expert group produced a consensus-based core set of data elements that allow the study of current and future contraceptives. These data elements influence patient and provider decisions about treatments and include important outcomes related to safety and effectiveness of these medical devices, which may benefit other women's health stakeholders.

Developing minimum core data structure for the obesity devices Coordinated Registry Network (CRN).

Obesity continues to be a major public health issue, with more than two-thirds of adults in the USA categorized as overweight or obese. Bariatric surgery is effective and yields durable weight loss; however, few qualified candidates choose to undergo surgical treatment. Less-invasive alternatives to bariatric surgery are being developed to bridge the treatment gap. Recognizing the burden of conducting pivotal clinical trials and traditional post-approval studies for medical devices, the Food and Drug Administration (FDA) Center for Devices and Radiological Health has encouraged the development of real-world data content and quality that is sufficient to provide evidence for Total Product Life Cycle medical device evaluation. A key first step is to establish a minimum core data structure that provides a common lexicon for endoscopic obesity devices and its corresponding interoperable data elements. Such a structure would facilitate data capture across existing workflow with a 'coordinated registry network' capability. On July 29, 2016, a workshop entitled, 'GI Coordinated Registry Network: A Case for Obesity Devices' was held at the FDA White Oak Campus by the Medical Device Epidemiology Network public-private partnership and FDA to initiate the work of developing a common lexicon and core data elements in the metabolic device space, which marked the inauguration of the Gastrointestinal Coordinated Registry Network project. Several work groups were subsequently formed to address clinical issues, data quality issues, registry participation, and data sharing.

Development of a coordinated registry network for pelvic organ prolapse technologies.

Objectives: The accumulation of data through a prospective, multicenter Coordinated Registry Network (CRN) could be a robust and cost-effective way to gather real-world evidence on the performance of pelvic organ prolapse (POP) technologies for device-based and intervention-based studies. To develop the CRN, a group of POP experts consisting of representatives from professional societies, the Food and Drug Administration, academia, industry, and the patient community, was convened to discuss the role and feasibility of the CRN and to identify the core data elements important to assess POP technologies. Design: A Delphi method approach was employed to achieve consensus on a core minimum dataset for the CRN. A series of surveys were sent to the panel and answered by each expert anonymously and individually. Results from the surveys were collected, collated, and analyzed by the study design team from Weill Cornell Medicine. Questions for the next round were based on the analysis process and discussed with group members via conference call. This process was repeated twice over a 6-month time period during which consensus was achieved. Results: Twenty-one experts participated in the effort and proposed 120 data elements. Participation rates in the first and second round of the Delphi survey were 95.2% and 71.4%, respectively. The working group reached final consensus among responders on 90 data elements capturing relevant general medical and surgical history, procedure and discharge, short-term and long-term follow-up, device factors, and surgery and surgeon factors. Conclusions: The CRN successfully developed a set of core data elements to support the study of POP technologies through convening an expert panel on POP technologies and using the Delphi method. These standardized data elements have the potential to influence patient and provider decisions about treatments and include important outcomes related to efficacy and safety.

Advancing the Real-World Evidence for Medical Devices through Coordinated Registry Networks.

Objectives: Generating and using real-world evidence (RWE) is a pragmatic solution for evaluating health technologies. RWE is recognized by regulators, health technology assessors, clinicians, and manufacturers as a valid source of information to support their decision-making. Well-designed registries can provide RWE and become more powerful when linked with electronic health records and administrative databases in coordinated registry networks (CRNs). Our objective was to create a framework of maturity of CRNs and registries, so guiding their development and the prioritization of funding. Design setting and participants: We invited 52 stakeholders from diverse backgrounds including patient advocacy groups, academic, clinical, industry and regulatory experts to participate on a Delphi survey. Of those invited, 42 participated in the survey to provide feedback on the maturity framework for CRNs and registries. An expert panel reviewed the responses to refine the framework until the target consensus of 80% was reached. Two rounds of the Delphi were distributed via Qualtrics online platform from July to August 2020 and from October to November 2020. Main outcome measures: Consensus on the maturity framework for CRNs and registries consisted of seven domains (unique device identification, efficient data collection, data quality, product life cycle approach, governance and sustainability, quality improvement, and patient-reported outcomes), each presented with five levels of maturity. Results: Of 52 invited experts, 41 (79.9%) responded to round 1; all 41 responded to round 2; and consensus was reached for most domains. The expert panel resolved the disagreements and final consensus estimates ranged from 80.5% to 92.7% for seven domains. Conclusions: We have developed a robust framework to assess the maturity of any CRN (or registry) to provide reliable RWE. This framework will promote harmonization of approaches to RWE generation across different disciplines and health systems. The domains and their levels may evolve over time as new solutions become available.

Two-Year Revision Rates in Total Ankle Replacement Versus Ankle Arthrodesis: A Population-Based Propensity-Score-Matched Comparison from New York State and California.

The aim of this study was to compare outcomes between total ankle replacement (TAR) and ankle arthrodesis (AA) for ankle osteoarthritis using real-world data. Methods: We used longitudinal claims data from New York State from October 2015 to December 2018, and from California from October 2015 to December 2017. The primary outcome was revision. Secondary outcomes were in-hospital complications and below-the-knee amputation. Propensity-score matching adjusted for differences in baseline characteristics. To determine predictors of the main outcome, each group was analyzed using multivariable Cox regressions. Results: There were 1,477 TAR procedures (50.2%) and 1,468 AA procedures (49.8%). Patients undergoing TAR were less likely to belong to a minority group and had fewer comorbidities compared with those undergoing AA. Crude analyses indicated that the TAR group had a lower risk of revision (5.4% versus 9.1%), in-hospital complications (<1% versus 1.8%), and below-the-knee amputation (<1% versus 4.9%) (p < 0.001 for all). However, in the propensity-score-matched analysis, the risk of revision was no longer significantly lower (TAR, 5.6% versus AA, 7.6%; p = 0.16). In the multivariable analyses, older age was predictive of a lower risk of revision after TAR (hazard ratio [HR], 0.96 [95% confidence interval (CI), 0.93 to 1.00]), but age was not predictive of revision after AA (HR, 0.99 [95% CI, 0.97 to 1.01]). Female patients were less likely to undergo revision after AA (HR, 0.61 [95% CI, 0.39 to 0.96]), but sex was not predictive of revision after TAR (HR, 0.90 [95% CI, 0.51 to 1.60]). Conclusions: The 2-year adjusted revision risk was 5.6% after TAR and 7.6% after AA. This difference did not reach significance. Older age was a predictor of lower revision risk after TAR. Men had a higher risk of revision than women after AA. The number of TAR procedures has now caught up with the number of AA procedures. Level of Evidence: Therapeutic Level III. See Instructions for Authors for a complete description of levels of evidence.

Creation of objective performance criteria among medical devices.

Objectives: Objective performance criteria (OPC) may serve as a tool to expedite the approval process and continue active surveillance of class III medical devices. Thus far, published guidance on the creation of OPC has been clinical area-specific. This study aimed to capture reflections from key stakeholders on the creation of OPC that may serve as a precursor for a formalized conceptual framework within the USA. Design: Reflections from key stakeholders and guidance from an advisory committee were captured to gain an understanding of the elements that are crucial to the generation of OPC. Setting: A non-probability sampling method using the purposive sampling strategy was employed to identify relevant stakeholders for engagement in semi-structured, open-ended, concept elicitation discussions. Participants: Stakeholders involved in the generation of OPC. Main outcome measures: Elements and themes regarding the priorities of, experiences with, roles within and perceived challenges associated with OPC creation captured through a phenomenological approach. Results: A total of 27 participants were engaged to represent the following contributors: representatives of registries, health systems, health technology assessment bodies, clinicians, device application reviewers, payers, patients, patient representatives, patient caregivers, device manufacturers, data coordinators, data analysts and data informaticians. Consensus was achieved on the five core elements: (1) identification of medical devices, (2) engagement of key stakeholders, (3) selection of data source, (4) performance of appropriate statistical analyses and (5) reporting of findings. The engagement of key stakeholders (38%) was cited most frequently as the most important core element. Access to meaningful and high-quality data sources (47%) was the most frequently mentioned challenge. Conclusions: The reflections from the participants identified five elements to be considered when generating an OPC within class III medical devices and may provide the needed foundation for the development of official guidance on OPC generation.