Validation of algorithms in studies based on routinely collected health data: general principles.

Clinicians, researchers, regulators, and other decision-makers increasingly rely on evidence from real-world data (RWD), including data routinely accumulating in health and administrative databases. RWD studies often rely on algorithms to operationalize variable definitions. An algorithm is a combination of codes or concepts used to identify persons with a specific health condition or characteristic. Establishing the validity of algorithms is a prerequisite for generating valid study findings that can ultimately inform evidence-based health care. This paper aims to systematize terminology, methods, and practical considerations relevant to the conduct of validation studies of RWD-based algorithms. We discuss measures of algorithm accuracy; gold/reference standard; study size; prioritizing accuracy measures; algorithm portability; and implication for interpretation. Information bias is common in epidemiologic studies, underscoring the importance of transparency in decisions regarding choice and prioritizing measures of algorithm validity. The validity of an algorithm should be judged in the context of a data source, and one size does not fit all. Prioritizing validity measures within a given data source depends on the role of a given variable in the analysis (eligibility criterion, exposure, outcome or covariate). Validation work should be part of routine maintenance of RWD sources.

The association of major adverse limb events and combination stent and atherectomy in patients undergoing revascularization for lower extremity peripheral artery disease.

BACKGROUND: The effectiveness of combined atherectomy and stenting relative to use of each procedure alone for the treatment of lower extremity peripheral artery disease has not been evaluated. AIMS: The objective of this study was to evaluate the short- and long-term major adverse limb event (MALE) following the receipt of stenting, atherectomy, and the combination of stent and atherectomy. METHODS: A retrospective cohort of patients undergoing atherectomy, stent, and combination stent atherectomy for lower extremity peripheral artery disease was derived from the Vascular Quality Initiative (VQI) data set. The primary outcome was MALE and was assessed in the short-term and long-term. Short-term MALE was assessed immediately following the procedure to discharge and estimated using logistic regression. Long-term MALE was assessed after discharge to end of follow-up and estimated using the Fine-Gray subdistribution hazard model. RESULTS: Among the 46,108 included patients, 6896 (14.95%) underwent atherectomy alone, 35,774 (77.59%) received a stent, and 3438 (7.5%) underwent a combination of stenting and atherectomy. The adjusted model indicated a significantly higher odds of short-term MALE in the atherectomy group (OR = 1.35; 95% confidence interval [CI]:1.16-1.57), and not significantly different odds (OR = 0.93; 95% CI:0.77-1.13) in the combination stent and atherectomy group when compared to stenting alone. With regard to long-term MALE, the model indicated that the likelihood of experiencing the outcome was slightly lower (HR = 0.90; 95% CI:0.82-0.98) in the atherectomy group, and not significantly different (HR = 0.92; 95% CI:0.82-1.04) in the combination stent and atherectomy group when compared to the stent group. CONCLUSIONS: Patients in the VQI data set who received combination stenting and atherectomy did not experience significantly different rates of MALE when compared with stenting alone. It is crucial to consider and further evaluate the influence of anatomical characteristics on treatment strategies and potential differential effects of comorbidities and other demographic factors on the short and long-term MALE risks.

Editor's Choice - Real World Study of Mortality After the Use of Paclitaxel Coated Devices in Peripheral Vascular Intervention.

OBJECTIVE: This observational cohort study examined outcomes after peripheral vascular intervention (PVI) with paclitaxel coated devices (PCD) and non-PCD, and evaluated heterogeneity of treatment effect in populations of interest. METHODS: The study included patients undergoing percutaneous transluminal angioplasty and or stent placement between 1 October 2015 and 31 December 2018 in the Vascular Quality Initiative Registry linked to Medicare claims. It determined differences in patient mortality and ipsilateral major amputation after PVI with PCD and non-PCD using Kaplan-Meier analyses and Cox regressions with inverse probability weighting in three cohorts: (A) patients treated for femoropopliteal or infrapopliteal occlusive disease with or without any other concurrent treatment (n = 11 452); (B) those treated for isolated superficial femoral or popliteal artery disease (n = 5 519); and (C) patients with inclusion criteria designed to approximate RCT populations (n = 2 278). RESULTS: The mean age of patients was 72.3 (SD = 10.9) years, and 40.6% were female. In cohort A, patients receiving PCD had a lower mortality rate (HR 0.88, 95% CI 0.79 - 0.98) than those receiving non-PCD. There was no significant difference in mortality between groups in cohort B (HR 0.91, 95% CI 0.80 - 1.04) and cohort C (HR 1.10, 95% CI 0.84 - 1.43). Patients receiving PCD did not have a significantly elevated risk of major amputation compared with those receiving non-PCD (cohort A: HR 0.84, 95% CI 0.70 - 1.00; cohort B: HR 0.84, 95% CI 0.67 - 1.06; and cohort C: HR 1.05, 95% CI 0.51 - 2.14). CONCLUSION: No increased patient mortality or major amputation was found at three years after PVI with PCD vs. non-PCD in this large, linked registry claims study, after accounting for heterogeneity of treatment effect by population. The analysis and results from three cohorts intended to mirror the cohorts of previous studies provide robust and niche real world evidence on PCD safety and help to understand and reconcile previously discrepant findings.

Venous Access: National Guideline and Registry Development (VANGUARD): Advancing Patient-Centered Venous Access Care Through the Development of a National Coordinated Registry Network.

There are over 8 million central venous access devices inserted each year, many in patients with chronic conditions who rely on central access for life-preserving therapies. Central venous access device-related complications can be life-threatening and add tens of billions of dollars to health care costs, while their incidence is most likely grossly mis- or underreported by medical institutions. In this communication, we review the challenges that impair retention, exchange, and analysis of data necessary for a meaningful understanding of critical events and outcomes in this clinical domain. The difficulty is not only with data extraction and harmonization from electronic health records, national surveillance systems, or other health information repositories where data might be stored. The problem is that reliable and appropriate data are not recorded, or falsely recorded, at least in part because policy, payment, penalties, proprietary concerns, and workflow burdens discourage completeness and accuracy. We provide a roadmap for the development of health care information systems and infrastructure that address these challenges, framed within the context of research studies that build a framework of standardized terminology, decision support, data capture, and information exchange necessary for the task. This roadmap is embedded in a broader Coordinated Registry Network Learning Community, and facilitated by the Medical Device Epidemiology Network, a Public-Private Partnership sponsored by the US Food and Drug Administration, with the scope of advancing methods, national and international infrastructure, and partnerships needed for the evaluation of medical devices throughout their total life cycle.

Contemporary incidence, outcomes, and survival associated with endovascular aortic aneurysm repair conversion to open repair among Medicare beneficiaries.

OBJECTIVE: The widespread application of endovascular abdominal aortic aneurysm repair (EVAR) has ushered in an era of requisite postoperative surveillance and the potential need for reintervention. The national prevalence and results of EVAR conversion to open repair, however, remain poorly defined. The purpose of this analysis was to define the incidence of open conversion and its associated outcomes. METHODS: The SVS Vascular Quality Initiative EVAR registry linked to Medicare claims via Vascular Implants Surveillance and Interventional Outcomes Network was queried for open conversions after initial EVAR procedures from 2003 to 2016. Cumulative conversion incidence within up to 5 years after EVAR and outcomes after open intervention were determined. Multivariable logistic regressions were used to identify independent predictors of conversion and mortality. RESULTS: Among 15,937 EVAR patients, 309 (1.9%) underwent an open conversion: 43% (n = 132) early (<30 days) and 57% (n = 177) late (>30 days). The longitudinally observed rate of conversion was constant over time, as well as by geographic region. Independent predictors of conversion included female sex (hazard ratio [HR], 1.49; P < .001), aneurysm diameter or more than 6.0 cm at the time of index EVAR (HR, 1.74; P < .001), nonelective repair (compared with elective presentation: HR, 1.72; P < .001), and aortouni-iliac repairs (HR, 2.19; P < .001). In contrast, adjunctive operative procedures such as endo-anchors or cuff extensions (HR, 0.62; P = .06) were protective against long-term conversion. Both early (HR, 1.6; P < .001) and late (HR, 1.26; P = .07) open conversions were associated with significant 30-day (total cohort, 15%) and 1-year mortality (total cohort, 25%). Patients undergoing open conversion experienced high rates of 30-day readmission (42%) and cardiac (45%), renal (32%), and pulmonary (30%) complications. CONCLUSIONS: This large, registry-based analysis is among the first to document the incidence and outcomes for open conversion after EVAR in a national cohort with long-term follow-up. Importantly, women, patients with large aneurysms, and complex anatomy, as well as urgent or emergent EVARs are at an increased risk for open conversion. It seems that more conversions are performed in the early postoperative period, despite perceptions that conversion is a delayed phenomenon. In all instances, conversion is associated with significant morbidity and mortality and highlights the importance of appropriate patient selection at the time of index EVAR.

Registry Assessment of Peripheral Interventional Devices objective performance goals for superficial femoral and popliteal artery peripheral vascular interventions.

BACKGROUND: The Superficial Femoral Artery-Popliteal EvidencE Development Study Group developed contemporary objective performance goals (OPGs) for peripheral vascular interventions (PVI) for superficial femoral artery (SFA)-popliteal artery disease using the Registry Assessment of Peripheral Interventional Devices. METHODS: The Society for Vascular Surgery Vascular Quality Initiative PVI registry from January 2010 to October 2016 was used to develop OPGs based on SFA-popliteal procedures (n = 21,377) for intermittent claudication and critical limb ischemia (CLI). OPGs included 1-year rates for target lesion revascularization (TLR), major amputation, and 1 and 4-year survival rates. OPGs were calculated for the SFA and popliteal arteries and stratified by four treatments: angioplasty alone (percutaneous transluminal angioplasty [PTA]), self-expanding stenting, atherectomy, and any treatment type. Outcomes were illustrated by unadjusted Kaplan-Meier analyses. RESULTS: Cohorts included PTA (n = 7505), stenting (n = 9217), atherectomy (n = 2510) and any treatment (n = 21,377). The mean age was 69 years, 58% were male, 79% were White, and 52% had CLI. The freedom from TLR OPGs at 1 year in the SFA were 80.3% (PTA), 83.2% (stenting), 83.9% (atherectomy), and 81.9% (any treatments). The freedom from TLR OPGs at 1 year in the popliteal were 81.3% (PTA), 81.3% (stenting), 80.2% (atherectomy), and 81.1% (any treatments). The freedom from major amputation OPGs at 1 year after SFA PVI were 93.4% (PTA), 95.7% (stenting), 95.1% (atherectomy), and 94.8% (any treatments). The freedom from major amputation OPG at 1 year after popliteal PVI were 90.5% (PTA), 93.7% (stenting), 91.8% (atherectomy), and 91.8%, (any treatments). The 4-year survival OPGs after SFA PVI were 76% (PTA), 80% (stenting), 82% (atherectomy), and 79% (any treatments), and for the popliteal artery were 72% (PTA), 77% (stenting), 82% (atherectomy), and 75% (any treatment). On a multivariable analysis, which included patient-level, leg-level, and lesion-level covariates, CLI was the single independent factor associated with increased TLR, amputation, and mortality. CONCLUSIONS: The Superficial Femoral Artery-Popliteal EvidencE Development OPGs define a new, contemporary benchmark for SFA-popliteal interventions using a large subset of real-world evidence to inform more efficient peripheral device clinical trial designs to support regulatory and clinical decision-making. It is appropriate to discuss proposals intended for regulatory approval with the US Food and Drug Administration to refine the OPG to match the specific trial population. The OPGs may be updated using coordinated registry networks to assess long-term real-world device performance.

Using propensity scores to estimate effects of treatment initiation decisions: State of the science.

Confounding can cause substantial bias in nonexperimental studies that aim to estimate causal effects. Propensity score methods allow researchers to reduce bias from measured confounding by summarizing the distributions of many measured confounders in a single score based on the probability of receiving treatment. This score can then be used to mitigate imbalances in the distributions of these measured confounders between those who received the treatment of interest and those in the comparator population, resulting in less biased treatment effect estimates. This methodology was formalized by Rosenbaum and Rubin in 1983 and, since then, has been used increasingly often across a wide variety of scientific disciplines. In this review article, we provide an overview of propensity scores in the context of real-world evidence generation with a focus on their use in the setting of single treatment decisions, that is, choosing between two therapeutic options. We describe five aspects of propensity score analysis: alignment with the potential outcomes framework, implications for study design, estimation procedures, implementation options, and reporting. We add context to these concepts by highlighting how the types of comparator used, the implementation method, and balance assessment techniques have changed over time. Finally, we discuss evolving applications of propensity scores.

Registry-Based Prospective, Active Surveillance of Medical-Device Safety.

BACKGROUND: The process of assuring the safety of medical devices is constrained by reliance on voluntary reporting of adverse events. We evaluated a strategy of prospective, active surveillance of a national clinical registry to monitor the safety of an implantable vascular-closure device that had a suspected association with increased adverse events after percutaneous coronary intervention (PCI). METHODS: We used an integrated clinical-data surveillance system to conduct a prospective, propensity-matched analysis of the safety of the Mynx vascular-closure device, as compared with alternative approved vascular-closure devices, with data from the CathPCI Registry of the National Cardiovascular Data Registry. The primary outcome was any vascular complication, which was a composite of access-site bleeding, access-site hematoma, retroperitoneal bleeding, or any vascular complication requiring intervention. Secondary safety end points were access-site bleeding requiring treatment and postprocedural blood transfusion. RESULTS: We analyzed data from 73,124 patients who had received Mynx devices after PCI procedures with femoral access from January 1, 2011, to September 30, 2013. The Mynx device was associated with a significantly greater risk of any vascular complication than were alternative vascular-closure devices (absolute risk, 1.2% vs. 0.8%; relative risk, 1.59; 95% confidence interval [CI], 1.42 to 1.78; P<0.001); there was also a significantly greater risk of access-site bleeding (absolute risk, 0.4% vs. 0.3%; relative risk, 1.34; 95% CI, 1.10 to 1.62; P=0.001) and transfusion (absolute risk, 1.8% vs. 1.5%; relative risk, 1.23; 95% CI, 1.13 to 1.34; P<0.001). The initial alerts occurred within the first 12 months of monitoring. Relative risks were greater in three prespecified high-risk subgroups: patients with diabetes, those 70 years of age or older, and women. All safety alerts were confirmed in an independent sample of 48,992 patients from April 1, 2014, to September 30, 2015. CONCLUSIONS: A strategy of prospective, active surveillance of a clinical registry rapidly identified potential safety signals among recipients of an implantable vascular-closure device, with initial alerts occurring within the first 12 months of monitoring. (Funded by the Food and Drug Administration and others.).

The Vascular Implant Surveillance and Interventional Outcomes (VISION) Coordinated Registry Network: An effort to advance evidence evaluation for vascular devices.

The Vascular Implant Surveillance and Interventional Outcomes Network (VISION) is a Coordinated Registry Network (CRN) a member of Medical Device Epidemiology Network, a U.S. Food and Drug Administration (FDA)-supported global public-private partnership that seeks to advance the collection and use of real-world data to improve patient outcomes. The VISION CRN began in September 2015 and held its first strategic meeting on September 10, 2018, at the FDA headquarters in Silver Spring, Maryland. VISION is a collaboration of the Vascular Quality Initiative (VQI), the FDA, and other stakeholders. At this annual meeting, leaders from the FDA, VQI, industry representatives, population health researchers, and regulatory science experts gathered to discuss strategic goals and opportunities for VISION. One of the key focus areas for VISION is linkage of VQI registry data to Medicare, longitudinal data sources maintained by various states, and other relevant data sources, as a model for efficient, cost-saving, and effectual evidence generation and appraisal. This would provide the means to expand data collection, assess long-term procedural outcomes across the carotid, lower extremity, aortic, and venous intervention datasets, and execute registry-based trials through the CRN structure in an efficient, cost-effective manner. Looking forward, VISION strives to validate long-term outcome data in the VQI using industry datasets, in hopes of using CRNs to make device regulatory decisions. With the guidance of a steering committee, VISION will provide vascular surgeons, industry, and regulators the appropriate data to improve care for patients with vascular disease.

Claims-based cardiovascular outcome identification for clinical research: Results from 7 large randomized cardiovascular clinical trials.

BACKGROUND: Medicare insurance claims may provide an efficient means to ascertain follow-up of older participants in clinical research. We sought to determine the accuracy and completeness of claims- versus site-based follow-up with clinical event committee (+CEC) adjudication of cardiovascular outcomes. METHODS: We performed a retrospective study using linked Medicare and Duke Database of Clinical Trials data. Medicare claims were linked to clinical data from 7 randomized cardiovascular clinical trials. Of 52,476 trial participants, linking resulted in 5,839 (of 10,497 linkage-eligible) Medicare-linked trial participants with fee-for-service A and B coverage. Death, myocardial infarction (MI), stroke, and revascularization incidences were compared using Medicare inpatient claims only, site-reported events (+CEC) only, or a combination of the 2. Randomized treatment effects were compared as a function of whether claims-based, site-based (+CEC), or a combined system was used for event detection. RESULTS: Among the 5,839 study participants, the annual event rates were similar between claims- and site-based (+CEC) follow-up: death (overall rate 5.2% vs 5.2%; adjusted κ 0.99), MI (2.2% vs 2.3%; adjusted κ 0.96), stroke (0.7% vs 0.7%; adjusted κ 0.99), and any revascularization (7.4% vs 7.9%; adjusted κ 0.95). Of events detected by claims yet not reported by CEC, a minority were reported by sites but negatively adjudicated by CEC (39% of MIs and 18% of strokes). Differences in individual case concordance led to higher event rates when claims- and site-based (+CEC) systems were combined. Randomized treatment effects were similar among the 3 approaches for each outcome of interest. CONCLUSIONS: Claims- versus site-based (+CEC) follow-up identified similar overall cardiovascular event rates despite meaningful differences in the events detected. Randomized treatment effects were similar using the 2 methods, suggesting claims data could be used to support clinical research leveraging routinely collected data. This approach may lead to more effective evidence generation, synthesis, and appraisal of medical products and inform the strategic approaches toward the National Evaluation System for Health Technology.

SPARED Collaboration: Patient Selection for Partial Gland Ablation in Men with Localized Prostate Cancer.

PURPOSE: The SPARED CRN (Study of Prostate Ablation Related Energy Devices Coordinated Registry Network) is a private-public partnership between academic and community urologists, the FDA (U.S. Food and Drug Administration), the Medical Device Epidemiology Network and device manufacturers to examine the safety and effectiveness of technologies for partial gland ablation in men with localized prostate cancer. MATERIALS AND METHODS: We report on a recent workshop at the FDA with thought leaders to discuss a critical framework for partial gland ablation, focusing on patient selection, surgical planning, followup, study design and appropriate comparators in terms of adverse events and cancer control outcomes. We summarize salient points from experts in urology, oncology and epidemiology that were presented and discussed in an open forum. RESULTS: Given the challenges in achieving patient and physician equipoise to perform a randomized trial, as well as an inherent paradigm shift when comparing partial gland ablation (inability to assess prostate specific antigen recurrence) to whole gland treatments, the group focused on objective performance criteria and goals as a platform to guide the creation of single arm studies in the SPARED CRN. CONCLUSIONS: This summit lays the foundation for prospective, multi-center data collection and evaluation of novel medical devices and drug/device combinations for partial gland ablation.

Development of a Nationally Representative Coordinated Registry Network for Prostate Ablation Technologies.

PURPOSE: The accumulation of data through a prospective, multicenter coordinated registry network is a practical way to gather real world evidence on the performance of novel prostate ablation technologies. Urological oncologists, targeted biopsy experts, industry representatives and representatives of the FDA (Food and Drug Administration) convened to discuss the role, feasibility and important data elements of a coordinated registry network to assess new and existing prostate ablation technologies. MATERIALS AND METHODS: A multiround Delphi consensus approach was performed which included the opinion of 15 expert urologists, representatives of the FDA and leadership from high intensity focused ultrasound device manufacturers. Stakeholders provided input in 3 consecutive rounds with conference calls following each round to obtain consensus on remaining items. Participants agreed that these elements initially developed for high intensity focused ultrasound are compatible with other prostate ablation technologies. Coordinated registry network elements were reviewed and supplemented with data elements from the FDA common study metrics. RESULTS: The working group reached consensus on capturing specific patient demographics, treatment details, oncologic outcomes, functional outcomes and complications. Validated health related quality of life questionnaires were selected to capture patient reported outcomes, including the IIEF-5 (International Index of Erectile Function-5), the I-PSS (International Prostate Symptom Score), the EPIC-26 (Expanded Prostate Cancer Index Composite-26) and the MSHQ-EjD (Male Sexual Health Questionnaire for Ejaculatory Dysfunction). Group consensus was to obtain followup multiparametric magnetic resonance imaging and prostate biopsy approximately 12 months after ablation with additional imaging or biopsy performed as clinically indicated. CONCLUSIONS: A national prostate ablation coordinated registry network brings forth vital practice pattern and outcomes data for this emerging treatment paradigm in the United States. Our multiple stakeholder consensus identifies critical elements to evaluate new and existing energy modalities and devices.

Quantifying the utilization of medical devices necessary to detect postmarket safety differences: A case study of implantable cardioverter defibrillators.

PURPOSE: To estimate medical device utilization needed to detect safety differences among implantable cardioverter defibrillators (ICDs) generator models and compare these estimates to utilization in practice. METHODS: We conducted repeated sample size estimates to calculate the medical device utilization needed, systematically varying device-specific safety event rate ratios and significance levels while maintaining 80% power, testing 3 average adverse event rates (3.9, 6.1, and 12.6 events per 100 person-years) estimated from the American College of Cardiology's 2006 to 2010 National Cardiovascular Data Registry of ICDs. We then compared with actual medical device utilization. RESULTS: At significance level 0.05 and 80% power, 34% or fewer ICD models accrued sufficient utilization in practice to detect safety differences for rate ratios <1.15 and an average event rate of 12.6 events per 100 person-years. For average event rates of 3.9 and 12.6 events per 100 person-years, 30% and 50% of ICD models, respectively, accrued sufficient utilization for a rate ratio of 1.25, whereas 52% and 67% for a rate ratio of 1.50. Because actual ICD utilization was not uniformly distributed across ICD models, the proportion of individuals receiving any ICD that accrued sufficient utilization in practice was 0% to 21%, 32% to 70%, and 67% to 84% for rate ratios of 1.05, 1.15, and 1.25, respectively, for the range of 3 average adverse event rates. CONCLUSIONS: Small safety differences among ICD generator models are unlikely to be detected through routine surveillance given current ICD utilization in practice, but large safety differences can be detected for most patients at anticipated average adverse event rates.

Innovative postmarket device evaluation using a quality registry to monitor thoracic endovascular aortic repair in the treatment of aortic dissection.

OBJECTIVE: United States Food and Drug Administration (FDA)-mandated postapproval studies have long been a mainstay of the continued evaluation of high-risk medical devices after initial marketing approval; however, these studies often present challenges related to patient/physician recruitment and retention. Retrospective single-center studies also do not fully represent the spectrum of real-world performance nor are they likely to have a sufficiently large enough sample size to detect important signals. In recent years, The FDA Center for Devices and Radiological Health has been promoting the development and use of patient registries to advance infrastructure and methodologies for medical device investigation. The FDA 2012 document, "Strengthening the National System for Medical Device Post-market Surveillance," highlighted registries as a core foundational infrastructure when linked to other complementary data sources, including embedded unique device identification. The Vascular Quality Initiative (VQI) thoracic endovascular aortic repair for type B aortic dissection project is an innovative method of using quality improvement registries to meet the needs of device evaluation after market approval. Here we report the organization and background of this project and highlight the innovation facilitated by collaboration of physicians, the FDA, and device manufacturers. METHODS: This effort used an existing national network of VQI participants to capture patients undergoing thoracic endovascular aortic repair for acute type B aortic dissection within a registry that aligns with standard practice and existing quality efforts. The VQI captures detailed patient, device, and procedural data for consecutive eligible cases under the auspices of a Patient Safety Organization (PSO). Patients were divided into a 5-year follow-up group (200 acute; 200 chronic dissections) and a 1-year follow-up group (100 acute; 100 chronic). The 5-year cohort required additional imaging details, and the 1-year group required standard VQI registry data entry. RESULTS: The sample size of patients in each of the 5-year acute and chronic dissection arms was achieved ≤24 months of project initiation, and data capture for the 1-year follow-up group is also nearly complete. Data completeness and follow-up has been excellent, and the two FDA-approved devices for dissection are equally represented. CONCLUSIONS: Although the completeness of long-term follow-up is yet to be determined, the rapidity of data collection supports the use of this construct for device assessment after market approval. The alignment of this effort with routine clinical practice and ongoing quality improvement initiatives is critical and has required minimal additional effort by practitioners, thus facilitating patient inclusion. Importantly, the success and development of this unique project has helped inform FDA strategy for future device evaluation after market approval.

A Systematic Review of Measurement Properties of Patient-Reported Outcome Measures Used in Patients Undergoing Total Knee Arthroplasty.

BACKGROUND: While clinical research on total knee arthroplasty (TKA) outcomes is prevalent in the literature, studies often have poor methodological and reporting quality. A high-quality patient-reported outcome instrument is reliable, valid, and responsive. Many studies evaluate these properties, but none have done so with a systematic and accepted method. The objectives of this study were to identify patient-reported outcome measures (PROMs) for TKA, and to critically appraise, compare, and summarize their psychometric properties using accepted methods. METHODS: MEDLINE, EMBASE, SCOPUS, Web of Science, PsycINFO, and SPORTDiscus were systematically searched for articles with the following inclusion criteria: publication before December 2014, English language, non-generic PRO, and evaluation in the TKA population. Methodological quality and evidence of psychometric properties were assessed with the COnsensus-based standards for the selection of health Status Measurement INstruments (COSMIN) checklist and criteria for psychometric evidence proposed by the COSMIN group and Terwee et al. RESULTS: One-hundred fifteen studies on 32 PROMs were included in this review. Only the Work, Osteoarthritis or joint-Replacement Questionnaire, the Oxford Knee Score, and the Western Ontario and McMaster Universities Arthritis Index had 4 or more properties with positive evidence. CONCLUSION: Most TKA PROMs have limited evidence for their psychometric properties. Although not all the properties were studied, the Work, Osteoarthritis or joint-Replacement Questionnaire, with the highest overall ratings, could be a useful PROM for evaluating patients undergoing TKA. The methods and reporting of this literature can improve by following accepted guidelines.

Predictable and SuStainable Implementation of National Cardiovascular Registries (PASSION) infrastructure: A think tank report from Medical Device Epidemiological Network Initiative (MDEpiNet).

The MDEpiNet is a public-private partnership between the US Food and Drug Administration's Center for Devices and Radiological Health and participating partners. The PASSION program is an MDEpiNet-sponsored program that aims to demonstrate the goals of MDEpiNet by using cardiovascular medical device registries to bridge evidence gaps across the medical device total product life cycle. To this end, a PASSION Think Tank meeting took place in October 2014 in Silver Spring, MD, to facilitate discussion between stakeholders about the successes, challenges, and future novel applications of medical device registries, with particular emphasis on identifying pilot projects. Participants spanned a broad range of groups including patients, device manufacturers, regulators, physicians/academicians, professional societies, providers, and payers. The meeting focus included 4 areas of cardiovascular medicine intended to cultivate interest in 4 MDEpiNet disease-specific/device-specific working groups: coronary intervention, electrophysiology, valvular disease, and peripheral vascular disease. In addition, more general issues applying to registry-based infrastructure and analytical methodologies for assessing device benefit/risk were considered to provide context for the working groups as PASSION programs going forward. This article summarizes the discussions at the meeting and the future directions of the PASSION program.

Long-Term Clinical Outcomes of Silzone Era St. Jude Medical Mechanical Heart Valves.

BACKGROUND AND AIM OF THE STUDY: Since the voluntary recall of St. Jude Medical (SJM) Silzone impregnated heart valves, no large-scale study has examined their long-term outcomes. METHODS: Using Medicare-linked records from the Society of Thoracic Surgeons Adult Cardiac Surgery Database (1993-2004), the clinical outcomes were evaluated through eight years among those patients who received SJM mechanical heart valves during the Silzone era (March 1998 to December 1999; n = 3,775), relative to those in both the pre-Silzone era (January 1993 to February 1998; n = 13,570) and the post-Silzone era (January 2000 to December 2004; n = 6,882). An inverse probability weighting was used to balance the observed differences in case mix. RESULTS: During the Silzone era, 79% of all implanted mechanical heart valves were manufactured by SJM. By eight years post-implantation, the most common adverse events in this Medicare-linked cohort (median age 71 years) were death (43.5%) and thromboembolism (14.7%), while valve reoperation (1.7%) and endocarditis (1.4%) were less common. Patients treated during the Silzone era experienced a lower associated risk of mortality to eight years than those in both the pre-Silzone era (adjusted hazards ratio (HR) 0.93, 95% confidence interval (CI) 0.88-0.98) and post-Silzone era (adjusted HR 0.92, CI 0.67-0.98), while the adjusted eight-year risks of reoperation, thromboembolism and endocarditis were similar across the three eras for the overall cohort and among both aortic valve and mitral valve patients. CONCLUSION: Medicare patients who received SJM mechanical heart valves during the Silzone era experienced similar clinical outcomes as those treated before or after the Silzone era. These data do not substantiate continued public health concerns associated with Silzone era valve prostheses among older individuals.

A Comprehensive Framework for Evaluating the Value Created by Real-World Evidence for Diverse Stakeholders: The Case for Coordinated Registry Networks.

OBJECTIVES: This manuscript presents a comprehensive framework for the assessment of the value of real-world evidence (RWE) in healthcare decision-making. While RWE has been proposed to overcome some limitations of traditional, one-off studies, no systematic framework exists to measure if RWE actually lowers the burden. This framework aims to fill that gap by providing conceptual approaches for evaluating the time and cost efficiencies of RWE, thus guiding strategic investments in RWE infrastructure. METHODS: The framework consists of four components: (114th Congress. 21st Century Cures Act.; 2015. https://www.congress.gov/114/plaws/publ255/PLAW-114publ255.pdf .) identification of stakeholders using and producing RWE, (National Health Council. Glossary of Patient Engagement Terms. Published 2019. Accessed May 18. 2021. https://nationalhealthcouncil.org/glossary-of-patient-engagement-terms/ .) understanding value propositions on how RWE can benefit stakeholders, (Center for Drug Evaluation and Research. CDER Patient-Focused Drug Development. U.S. Food & Drug Administration.) defining key performance indicators (KPIs), and (U.S. Department of Health and Human Services - Food and Drug Administration: Center for Devices and Radiological Health and Center for Biologics Evaluation and Research. Use of Real-World Evidence to Support Regulatory Decision-Making for Medical Devices - Guidance for Industry and Food and Drug Administration Staff. 2017. http://www.fda.gov/BiologicsBloodVaccines/GuidanceComplianceRegulatoryInformation/Guida .) establishing metrics and case studies to assess value. KPIs are categorized as 'better, faster, or cheaper" as an indicator of value: better focusing on high-quality actionable evidence; 'faster,' denoting time-saving in evidence generation, and 'cheaper,' emphasizing cost-efficiency decision compared to methodologies that do not involve data routinely collected in clinical practice. Metrics and relevant case studies are tailored based on stakeholder value propositions and selected KPIs that can be used to assess what value has been created by using RWE compared to traditional evidence-generation approaches and comparing different RWE sources. RESULTS: Operationalized through metrics and case studies drawn from the literature, the value of RWE is documented as improving treatment effect heterogeneity evaluation, expanding medical product labels, and expediting post-market compliance. RWE is also shown to reduce the cost and time required to produce evidence compared to traditional one-off approaches. An original example of a metric that measures the time saved by RWE methods to detect a signal of a product failure was presented based on analysis of the National Cardiovascular Disease Registry. CONCLUSIONS: The framework presented in this manuscript offers a comprehensive approach for evaluating the value of RWE, applicable to all stakeholders engaged in leveraging RWE for healthcare decision-making. Through the proposed metrics and illustrated case studies, valuable insights are provided into the heightened efficiency, cost-effectiveness, and improved decision-making within clinical and regulatory domains facilitated by RWE. While this framework is primarily focused on medical devices, it could potentially inform the determination of RWE value in other medical products. By discerning the variations in cost, time, and data utility among various evidence-generation methods, stakeholders are empowered to invest strategically in RWE infrastructure and shape future research endeavors.