Assessing adverse event reports of hysteroscopic sterilization device removal using natural language processing.

OBJECTIVE: To develop an annotation model to apply natural language processing (NLP) to device adverse event reports and implement the model to evaluate the most frequently experienced events among women reporting a sterilization device removal. METHODS: We included adverse event reports from the Manufacturer and User Facility Device Experience database from January 2005 to June 2018 related to device removal following hysteroscopic sterilization. We used an iterative process to develop an annotation model that extracts six categories of desired information and applied the annotation model to train an NLP algorithm. We assessed the model performance using positive predictive value (PPV, also known as precision), sensitivity (also known as recall), and F1 score (a combined measure of PPV and sensitivity). Using extracted variables, we summarized the reporting source, the presence of prespecified and other patient and device events, additional sterilizations and other procedures performed, and time from implantation to removal. RESULTS: The overall F1 score was 91.5% for labeled items and 93.9% for distinct events after excluding duplicates. A total of 16 535 reports of device removal were analyzed. The most frequently reported patient and device events were abdominal/pelvic/genital pain (N = 13 166, 79.6%) and device dislocation/migration (N = 3180, 19.2%), respectively. Of those reporting an additional sterilization procedure, the majority had a hysterectomy or salpingectomy (N = 7932). One-fifth of the cases that had device removal timing specified reported a removal after 7 years following implantation (N = 2444/11 293). CONCLUSIONS: We present a roadmap to develop an annotation model for NLP to analyze device adverse event reports. The extracted information is informative and complements findings from previous research using administrative data.

Data sources and applied methods for paclitaxel safety signal discernment.

Background: Following the identification of a late mortality signal, the Food and Drug Administration (FDA) convened an advisory panel that concluded that additional clinical study data are needed to comprehensively evaluate the late mortality signal observed with the use of drug-coated balloons (DCB) and drug-eluting stent (DES). The objective of this review is to (1) identify and summarize the existing clinical and cohort studies assessing paclitaxel-coated DCBs and DESs, (2) describe and determine the quality of the available data sources for the evaluation of these devices, and (3) present methodologies that can be leveraged for proper signal discernment within available data sources. Methods: Studies and data sources were identified through comprehensive searches. original research studies, clinical trials, comparative studies, multicenter studies, and observational cohort studies written in the English language and published from January 2007 to November 2021, with a follow-up longer than 36 months, were included in the review. Data quality of available data sources identified was assessed in three groupings. Moreover, accepted data-driven methodologies that may help circumvent the limitations of the extracted studies and data sources were extracted and described. Results: There were 39 studies and data sources identified. This included 19 randomized clinical trials, nine single-arm studies, eight registries, three administrative claims, and electronic health records. Methodologies focusing on the use of existing premarket clinical data, the incorporation of all contributed patient time, the use of aggregated data, approaches for individual-level data, machine learning and artificial intelligence approaches, Bayesian approaches, and the combination of various datasets were summarized. Conclusion: Despite the multitude of available studies over the course of eleven years following the first clinical trial, the FDA-convened advisory panel found them insufficient for comprehensively assessing the late-mortality signal. High-quality data sources with the capabilities of employing advanced statistical methodologies are needed to detect potential safety signals in a timely manner and allow regulatory bodies to act quickly when a safety signal is detected.

Is the FDA Failing Women?

Many devices in current use were marketed before the US Food and Drug Administration (FDA) began regulating devices in 1976. Thus, manufacturers of these devices were not required to demonstrate safety and effectiveness, which presents both clinical and ethical problem for patients, especially for women, as some of the most dangerous devices-such as implanted contraceptive devices-are used only in women. This article investigates whether and to what extent devices for women receive less rigorous scrutiny than devices for men. This article also suggests how the FDA Center for Devices and Radiological Health could more effectively ensure safety and effectiveness of devices that were marketed prior to 1976.

Reporting of Death in US Food and Drug Administration Medical Device Adverse Event Reports in Categories Other Than Death.

Importance: In the US, most postmarket medical device safety data are obtained through adverse event reports that are submitted to the US Food and Drug Administration (FDA)'s Manufacturer and User Facility Device Experience (MAUDE) database. Adverse event reports are classified by the reporter as injury, malfunction, death, or other. If the device may have caused or contributed to a death, or if the cause of death is unknown, the FDA requires that the adverse event be reported as a death. Objective: To determine the percentage of medical device adverse event reports submitted to the MAUDE database that were not classified as death even though the patient died. Design, Setting, and Participants: In this study, a natural language processing algorithm was applied to the MAUDE database, followed by manual text review, to identify reports in the injury, malfunction, other or missing categories that included at least 1 term that suggested a patient death, such as patient died or patient expired, from December 31, 1991, to April 30, 2020, for any medical device. Exposures: Manual review of a random sample of 1000 adverse event reports not classified as death and of selected reports for 62 terms that are associated with deaths but were not classified as death. Main Outcomes and Measures: Percentage of adverse event reports in which the patient was said to have died in the narrative section of the report but the reporter classified the report in a category other than death. Results: The terms in the natural language processing algorithm identified 290 141 reports in which a serious injury or death was reported. Of these, 151 145 (52.1%) were classified by the reporter as death and 47.9% were classified as malfunction, injury, other, or missing. For the overall sample, the percentage of reports with deaths that were not classified as deaths was 23% (95% CI, 20%-25%), suggesting that approximately 31 552 reports in our sample had deaths that were classified in other categories. The overall percentage of missed deaths, defined as the percentage of deaths that were classified in other categories, was 17% (95% CI, 16%-19%). Conclusions and Relevance: Many of the findings of this study suggest that many medical device adverse event reports in the FDA's MAUDE database that involved a patient death are classified in categories other than death. As the FDA only routinely reviews all adverse events that are reported as patient deaths, improving the accuracy of adverse event reporting may enhance patient safety.

Trends in FDA Adverse Events Reporting for Inferior Vena Cava Filters and Estimated Insertions in the US, 2016 to 2020.

This quality improvement study identifies adverse events for inferior vena cava filters and reports changes in adverse event reporting and estimated insertions between 2016 and 2020 in the US.