Developing a RadLex-Based Named Entity Recognition Tool for Mining Textual Radiology Reports: Development and Performance Evaluation Study.

BACKGROUND: Named entity recognition (NER) plays an important role in extracting the features of descriptions such as the name and location of a disease for mining free-text radiology reports. However, the performance of existing NER tools is limited because the number of entities that can be extracted depends on the dictionary lookup. In particular, the recognition of compound terms is very complicated because of the variety of patterns. OBJECTIVE: The aim of this study is to develop and evaluate an NER tool concerned with compound terms using RadLex for mining free-text radiology reports. METHODS: We leveraged the clinical Text Analysis and Knowledge Extraction System (cTAKES) to develop customized pipelines using both RadLex and SentiWordNet (a general purpose dictionary). We manually annotated 400 radiology reports for compound terms in noun phrases and used them as the gold standard for performance evaluation (precision, recall, and F-measure). In addition, we created a compound terms-enhanced dictionary (CtED) by analyzing false negatives and false positives and applied it to another 100 radiology reports for validation. We also evaluated the stem terms of compound terms by defining two measures: occurrence ratio (OR) and matching ratio (MR). RESULTS: The F-measure of cTAKES+RadLex+general purpose dictionary was 30.9% (precision 73.3% and recall 19.6%) and that of the combined CtED was 63.1% (precision 82.8% and recall 51%). The OR indicated that the stem terms of effusion, node, tube, and disease were used frequently, but it still lacks capturing compound terms. The MR showed that 71.85% (9411/13,098) of the stem terms matched with that of the ontologies, and RadLex improved approximately 22% of the MR from the cTAKES default dictionary. The OR and MR revealed that the characteristics of stem terms would have the potential to help generate synonymous phrases using the ontologies. CONCLUSIONS: We developed a RadLex-based customized pipeline for parsing radiology reports and demonstrated that CtED and stem term analysis has the potential to improve dictionary-based NER performance with regard to expanding vocabularies.

A Bayesian Network-Based Browsing Model for Patients Seeking Radiology-Related Information on Hospital Websites: Development and Usability Study.

BACKGROUND: An increasing number of people are visiting hospital websites to seek better services and treatments compared to the past. It is therefore important for hospitals to develop websites to meet the needs of their patients. However, few studies have investigated whether and how the current hospital websites meet the patient's needs. Above all, in radiation departments, it may be difficult for patients to obtain the desired information regarding modality and diagnosis because such information is subdivided when described on a website. OBJECTIVE: The purpose of this study is to suggest a hospital website search behavior model by analyzing the browsing behavior model using a Bayesian network from the perspective of one-to-one marketing. METHODS: First, we followed the website access log of Hokkaido University Hospital, which was collected from September 1, 2016, to August 31, 2017, and analyzed the access log using Google Analytics. Second, we specified the access records related to radiology from visitor browsing pages and keywords. Third, using these resources, we structured 3 Bayesian network models based on specific patient needs: radiotherapy, nuclear medicine examination, and radiological diagnosis. Analyzing each model, this study considered why some visitors could not reach their desired page and improvements to meet the needs of visitors seeking radiology-related information. RESULTS: The radiotherapy model showed that 74% (67/90) of the target visitors could reach their requested page, but only 2% (2/90) could reach the Center page where inspection information, one of their requested pages, is posted. By analyzing the behavior of the visitors, we clarified that connecting with the radiotherapy and radiological diagnosis pages is useful for increasing the proportion of patients reaching their requested page. CONCLUSIONS: We proposed solutions for patient web-browsing accessibility based on a Bayesian network. Further analysis is necessary to verify the accuracy of the proposed model in comparison to other models. It is expected that information provided on hospital websites will be improved using this method.

Cost-Effectiveness of a Continuous Glucose Monitoring Mobile App for Patients With Type 2 Diabetes Mellitus: Analysis Simulation.

BACKGROUND: Apps for real-time continuous glucose monitoring (CGM) on smartphones and other devices linked to CGM systems have recently been developed, and such CGM apps are also coming into use in Japan. In comparison with conventional retrospective CGM, the use of CGM apps improves patients' own blood glucose control, which is expected to help slow the progression of type 2 diabetes mellitus (DM) and prevent complications, but the effect of their introduction on medical costs remains unknown. OBJECTIVE: Our objective in this study was to perform an economic appraisal of CGM apps from the viewpoint of assessing public medical costs associated with type 2 DM, using the probability of developing type 2 DM-associated complications, and data on medical costs and utility value to carry out a medical cost simulation using a Markov model in order to ascertain the cost-effectiveness of the apps. METHODS: We developed a Markov model with the transition states of insulin therapy, nephrosis, dialysis, and cardiovascular disease, all of which have a major effect on medical costs, to identify changes in medical costs and utility values resulting from the introduction of a CGM app and calculated the incremental cost-effectiveness ratio (ICER). RESULTS: The ICER for CGM app use was US $33,039/quality-adjusted life year (QALY). CONCLUSIONS: Sensitivity analyses showed that, with the exception of conditions where the transition probability of insulin therapy, utility value, or increased medical costs increases, the ICER for the introduction of CGM apps was below the threshold of US $43,478/QALY used by the Central Social Insurance Medical Council. Our results provide basic data on the cost-effectiveness of introducing CGM apps, which are currently starting to come into use.

ADEpedia-on-OHDSI: A next generation pharmacovigilance signal detection platform using the OHDSI common data model.

OBJECTIVE: Supplementing the Spontaneous Reporting System (SRS) with Electronic Health Record (EHR) data for adverse drug reaction detection could augment sample size, increase population heterogeneity and cross-validate results for pharmacovigilance research. The difference in the underlying data structures and terminologies between SRS and EHR data presents challenges when attempting to integrate the two into a single database. The Observational Health Data Sciences and Informatics (OHDSI) collaboration provides a Common Data Model (CDM) for organizing and standardizing EHR data to support large-scale observational studies. The objective of the study is to develop and evaluate an informatics platform known as ADEpedia-on-OHDSI, where spontaneous reporting data from FDA's Adverse Event Reporting System (FAERS) is converted into the OHDSI CDM format towards building a next generation pharmacovigilance signal detection platform. METHODS: An extraction, transformation and loading (ETL) tool was designed, developed, and implemented to convert FAERS data into the OHDSI CDM format. A comprehensive evaluation, including overall ETL evaluation, mapping quality evaluation of drug names to RxNorm, and an evaluation of transformation and imputation quality, was then performed to assess the mapping accuracy and information loss using the FAERS data collected between 2012 and 2017. Previously published findings related to vascular safety profile of triptans were validated using ADEpedia-on-OHDSI in pharmacovigilance research. For the triptan-related vascular event detection, signals were detected by Reporting Odds Ratio (ROR) in high-level group terms (HLGT) level, high-level terms (HLT) level and preferred term (PT) level using the original FAERS data and CDM-based FAERS respectively. In addition, six standardized MedDRA queries (SMQs) related to vascular events were applied. RESULTS: A total of 4,619,362 adverse event cases were loaded into 8 tables in the OHDSI CDM. For drug name mapping, 93.9% records and 47.0% unique names were matched with RxNorm codes. Mapping accuracy of drug names was 96% based on a manual verification of randomly sampled 500 unique mappings. Information loss evaluation showed that more than 93% of the data is loaded into the OHDSI CDM for most fields, with the exception of drug route data (66%). The replication study detected 5, 18, 47 and 6, 18, 50 triptan-related vascular event signals in MedDRA HLGT level, HLT level, and PT level for the original FAERS data and CDM-based FAERS respectively. The signal detection scores of six standardized MedDRA queries (SMQs) of vascular events in the raw data study were found to be lower than those scores in the CDM study. CONCLUSION: The outcome of this work would facilitate seamless integration and combined analyses of both SRS and EHR data for pharmacovigilance in ADEpedia-on-OHDSI, our platform for next generation pharmacovigilance.

Developing a FHIR-based EHR phenotyping framework: A case study for identification of patients with obesity and multiple comorbidities from discharge summaries.

BACKGROUND: Standards-based clinical data normalization has become a key component of effective data integration and accurate phenotyping for secondary use of electronic healthcare records (EHR) data. HL7 Fast Healthcare Interoperability Resources (FHIR) is an emerging clinical data standard for exchanging electronic healthcare data and has been used in modeling and integrating both structured and unstructured EHR data for a variety of clinical research applications. The overall objective of this study is to develop and evaluate a FHIR-based EHR phenotyping framework for identification of patients with obesity and its multiple comorbidities from semi-structured discharge summaries leveraging a FHIR-based clinical data normalization pipeline (known as NLP2FHIR). METHODS: We implemented a multi-class and multi-label classification system based on the i2b2 Obesity Challenge task to evaluate the FHIR-based EHR phenotyping framework. Two core parts of the framework are: (a) the conversion of discharge summaries into corresponding FHIR resources - Composition, Condition, MedicationStatement, Procedure and FamilyMemberHistory using the NLP2FHIR pipeline, and (b) the implementation of four machine learning algorithms (logistic regression, support vector machine, decision tree, and random forest) to train classifiers to predict disease state of obesity and 15 comorbidities using features extracted from standard FHIR resources and terminology expansions. We used the macro- and micro-averaged precision (P), recall (R), and F1 score (F1) measures to evaluate the classifier performance. We validated the framework using a second obesity dataset extracted from the MIMIC-III database. RESULTS: Using the NLP2FHIR pipeline, 1237 clinical discharge summaries from the 2008 i2b2 obesity challenge dataset were represented as the instances of the FHIR Composition resource consisting of 5677 records with 16 unique section types. After the NLP processing and FHIR modeling, a set of 244,438 FHIR clinical resource instances were generated. As the results of the four machine learning classifiers, the random forest algorithm performed the best with F1-micro(0.9466)/F1-macro(0.7887) and F1-micro(0.9536)/F1-macro(0.6524) for intuitive classification (reflecting medical professionals' judgments) and textual classification (reflecting the judgments based on explicitly reported information of diseases), respectively. The MIMIC-III obesity dataset was successfully integrated for prediction with minimal configuration of the NLP2FHIR pipeline and machine learning models. CONCLUSIONS: The study demonstrated that the FHIR-based EHR phenotyping approach could effectively identify the state of obesity and multiple comorbidities using semi-structured discharge summaries. Our FHIR-based phenotyping approach is a first concrete step towards improving the data aspect of phenotyping portability across EHR systems and enhancing interpretability of the machine learning-based phenotyping algorithms.

Forecasting maldistribution of human resources for healthcare and patients in Japan: a utilization-based approach.

BACKGROUND: Hokkaido's demographic trend of population decrease with aging population is remarkable even in Japan. Although healthcare policy decision-makers need to appropriately allocate resources while grasping regional demands, not much is available on whether medical demand would increase or not for future. In addition, little is known about what impact will current situation have on future demand-supply balance and equality by regions. This study aims to support decision-making in human resource planning for coping with changing population structure by forecasting future demand, and evaluation those regional maldistributions. METHOD: We set patients with acute myocardial infarction or cerebral stroke, and all medical care as study subjects and analyzed for 2015, 2025, and 2035 in Hokkaido and each Secondary Medical Care Area. We used a utilization-based approach to estimate the healthcare supply-demand balance in the future. Moreover, we evaluated the regional maldistribution of demand-supply balance by calculating Herfindahl-Hirschman Index, Gini Coefficients, the number of physicians/specialists per patient. Moreover, we conducted sensitivity analysis to evaluation impact on aspects of demand-supply balance by uncertainty of utilization for future. RESULTS: Our results displayed that concentration of patients will progress, while regional distribution will shrink in all subject. However, from comparison based on all medical care, Gini Coefficients of acute myocardial infarction and cerebral stroke has always been high. This suggest that the resource allocation of them has room for improvement. In addition, our analysis showed the change in this balance will differ in each region in the future. Moreover, demographic change will not consistent with the number of patient change from 2015 to 2035. CONCLUSION: These results suggest policy planners should use the number of patient by disease, by region as indicator of demand, instead of provider-to-population ratios being in use today. The result of our sensitivity analysis show two findings. First, the range of each indicator have possible for future. Second, increase of utilization, for instance lowing barrier in the use by development operation of patient transportation in AMI/CS, would improve maldistribution of opportunity for resident to get emergency medical services.

Where Does Telemedicine Achieve a Cost Reduction Effect? Cost Minimization Analysis of Teleradiology Services in Japan.

Background: Telemedicine as a technology is expected to resolve issues such as doctor shortages and disparities in medical services. However, high costs of system installation and maintenance inhibit its widespread use.Introduction: This study involved a cost minimization analysis for installation of a teleradiology system in the Hokkaido prefecture of Japan. Conditions under which system utilization is cost-efficient and system utilization is effective for cost reduction were analyzed.Materials and Methods: A cost minimization analysis was conducted using three geospatial points of 50, 100, and 200 km from Sapporo city, the prefectural capital of Hokkaido, assuming a central imaging diagnosis center in Sapporo. The analysis was conducted from the standpoint of both patients and requesting hospitals.Results: From the patient's standpoint, a cost reduction effect was observed at all three distances from system installation. In contrast, from the hospital's standpoint, a cost reduction effect was found only when teleradiology examination was conducted from a distance of at least 100 km from Sapporo.Discussion: Results show that the cost reduction effect for patients increased as the travel distance increased. Although the teleradiology service is beneficial for a wide range of patients, the financial burden on requesting hospitals is significant.Conclusions: The following conditions were found necessary to reduce the requesting hospital's financial burden: the hospital should be far from the imaging diagnosis center, an inexpensive system is to be selected, and the system needs to be utilized continuously.

[Appling Text-mining to Extracting Technical Terms from Textbooks-Toward Updating the Terminology in the Field of Radiology Technology].

Although terminology requires continuous consideration of recorded technical terms, extracting these terms manually is difficult, because the number of recorded terms is constantly increasing. Text-mining acquires information from numerous documents, and is capable of extracting technical terms. The purpose of this study is to extract candidate terms using text-mining toward updating the terminology of Japanese society of radiological technology (JSRT). First, the subjects for this study were textbooks published by the JSRT, and morphological analysis was conducted, which is an analysis to break the books up into meaningful words. Additionally, index terms of textbooks were extracted. Second, we observed overlaps between the JSRT technical terms and the terms obtained from the morphological analysis and the indexes of textbooks and the extracted terms were absent in the JSRT terminology. The overlap was 53.6% (3090/5770 terms). The terms, "imaging technology for magnetic resonance" and "information and system in radiological technology" were missing from the JSRT terminology. From these results, it was estimated that half number of the JSRT technical terms were changing with time. This study demonstrated that text mining showed the differences between old and new technical terms.

[Comparison of Japanese Notation and Meanings among Three Terminologies in Radiological Technology Domain].

PURPOSE: The purpose of this study is to investigate the differences in the notation of technical terms and their meanings among three terminologies in Japanese radiology-related societies. MATERIALS AND METHODS: The three terminologies compared in this study were "radiological technology terminology" and its supplement published by the Japan Society of Radiological Technology, "medical physics terminology" published by the Japan Society of Medical Physics, and "electric radiation terminology" published by the Japan Radiological Society. Terms were entered into spreadsheets and classified into the following three categories: Japanese notation, English notation, and meanings. In the English notation, terms were matched to character strings in the three terminologies and were extracted and compared. The Japanese notations were compared among three terminologies, and the difference between the meanings of the two terminologies radiological technology terminology and electric radiation terminology were compared. RESULTS AND DISCUSSION: There were a total of 14,982 terms in the three terminologies. In English character strings, 2,735 terms were matched to more than two terminologies, with 801 of these terms matched to all the three terminologies. Of those terms in English character strings matched to three terminologies, 752 matched to Japanese character strings. Of the terms in English character strings matched to two terminologies, 1,240 matched to Japanese character strings. With regard to the meanings category, eight terms had mismatched meanings between the two terminologies. For these terms, there were common concepts between two different meaning terms, and it was considered that the derived concepts were described based on domain.

Development of a Knowledge-based Application Utilizing Ontologies for the Continuing Site-specific JJ1017 Master Maintenance.

The purpose of this study was to develop the JJ1017 Knowledge-based Application (JKA) to support the continuing maintenance of a site-specific JJ1017 master defined by the JJ1017 guideline as a standard radiologic procedure master for medical information systems that are being adopted by some medical facilities in Japan. The method consisted of the following three steps: (1) construction of the JJ1017 Ontology (JJOnt) as a knowledge base using the Hozo (an environment for building/using ontologies); (2) development of modules (operation, I/O, graph modules) that are required to continue the maintenance of a site-specific JJ1017 master; and (3) unit testing of the JKA that consists of the JJOnt and the modules. As a result, the number of classes included in the JJOnt was 21,697. Within the radiologic procedure classes included in the above, the ratio of a JJ1017 master code for an external beam radiotherapy was the highest (51%). In unit testing of the JKA, we checked the main operations (e.g., keyword search of a JJ1017 master code/code meaning, editing the description of classes, etc.). The JJOnt is a knowledge base for implementing features that medical technologists find necessary in medical information systems. To enable medical technologists to exchange/retrieve semantically accurate information while using medical information systems in the future, we expect the JKA to support the maintenance and improvement of the site-specific JJ1017 master.

[Classifying Ontology towards an Application for Radiological Technology].

This study aims to grasp the target area of the literature on ontology and to apply it in radiological technology. We used Google scholar to search the literature containing the keyword "ontology". Our search identified and extracted 162,381 words from 29 manuscripts and used the 8,706 nouns excluding duplicates as individual variable. Using a cluster analysis, we categorized the documents to one of the following five classifications: (1) "Systematization of vocabulary by text mining", (2) "Hierarchy of language information", (3) "Conceptualization of situation", (4) "Standardization of lexical information", and (5) "Visualization of the concepts related to the problem". We propose that the terminologies in (2), (4), and (5) cluster can be used in radiological technology field.

[An analysis of existing terminology towards constructing ontology in the field of the radiological technology].

PURPOSE: In 1994, Japanese Society of Radiological Technology (JSRT) constructed the lexicon in the field of radiologic technology. However, recently, latest lexicon is not updated yet. The purpose of this article is to compare the terminologies in clinical medicine with the others and to consider reconstructing the lexicon in the radiological technology. MATERIALS AND METHODS: Our study selected three categories from the database of the academic society. These three groups were Clinical medicine (hereafter CM, 167 societies, includes JSRT), Psychology / Education (hereafter P/E, 104 societies), and Comprehensive synthetic engineering (hereafter CSE, 40 societies). First, all societies were surveyed to know whether there were any lexicon in their official website. Second, these terminologies were surveyed on the following criteria: (a) Media of lexicon, (b) Number of terms, (c) File type of lexicon, (d) Terms translated into English, (e) Way of searching terms, and (f) Number of committees of the terminology. RESULTS: Lexicon in CM, P/E, and CSE had 20, 4, and 7. Compared with P/E and CSE, CM showed the following trends: (a) used electronic media frequently, (b) stored large number of terms (about 5,000 to 11,000), (c) enabled to download frequently, and (d) used the alphabet and Japanese syllabary order frequently. CONCLUSIONS: Compared with the lexicon of P/E and CSE, terminology in CM tended to adopt the electronic media of lexicon and to have large number of terms. Additionally, many lexicons were expressed in English terms along with Japanese terms. Following massive lexicon of SNOMED-CT and RadLex, it is necessary to consider applying the web-based term searching and an ontological technique to the lexicon of radiological technology.

Cancer Detection Rate and Abnormal Interpretation Rate of Prostate MRI in Patients With Low-Grade Cancer.

PURPOSE: The aim of this study was to evaluate the utility of cancer detection rate (CDR) and abnormal interpretation rate (AIR) in prostate MRI for patients with low-grade prostate cancer (PCa). METHODS: This three-center retrospective study included patients who underwent prostate MRI from 2017 to 2021 with known low-grade PCa (Gleason score 6) without prior treatment. Patient-level highest Prostate Imaging Reporting & Data System (PI-RADS®) score and pathologic diagnosis within 1 year after MRI were used to evaluate the diagnostic performance of prostate MRI in detecting clinically significant PCa (csPCa; Gleason score ≥ 7). The metrics AIR, CDR, and CDR adjusted for pathologic confirmation rate were calculated. Radiologist-level AIR-CDR plots were shown. Simulation AIR-CDR lines were created to assess the effects of different diagnostic performances of prostate MRI and the prevalence of csPCa. RESULTS: A total of 3,207 examinations were interpreted by 33 radiologists. Overall AIR, CDR, and CDR adjusted for pathologic confirmation rate at PI-RADS 3 to 5 (PI-RADS 4 and 5) were 51.7% (36.5%), 22.1% (18.8%), and 30.7% (24.6%), respectively. Radiologist-level AIR and CDR at PI-RADS 3 to 5 (PI-RADS 4 and 5) were in the 36.8% to 75.6% (21.9%-57.5%) range and the 16.3%-28.7% (10.9%-26.5%) range, respectively. In the simulation, changing parameters of diagnostic performance or csPCa prevalence shifted the AIR-CDR line. CONCLUSIONS: The authors propose CDR and AIR as performance metrics in prostate MRI and report reference performance values in patients with known low-grade PCa. There was variability in radiologist-level AIR and CDR. Combined use of AIR and CDR could provide meaningful feedback for radiologists to improve their performance by showing relative performance to other radiologists.

Cancer Detection Rate and Abnormal Interpretation Rate of Prostate MRI Performed for Clinical Suspicion of Prostate Cancer.

PURPOSE: To report cancer detection rate (CDR) and abnormal interpretation rate (AIR) in prostate MRI performed for clinical suspicion of prostate cancer (PCa). MATERIALS AND METHODS: This retrospective single-institution, three-center study included patients who underwent MRI for clinical suspicion of PCa between 2017 and 2021. Patients with known PCa were excluded. Patient-level Prostate Imaging-Reporting and Data System (PI-RADS) score was extracted from the radiology report. AIR was defined as number of abnormal MRI (PI-RADS score 3-5) / total number of MRIs. CDR was defined as number of clinically significant PCa (csPCa: Gleason score ≥7) detected at abnormal MRI / total number of MRI. AIR, CDR, and CDR adjusted for pathology confirmation rate were calculated for each of three centers and pre-MRI biopsy status (biopsy-naive and previous negative biopsy). RESULTS: A total of 9,686 examinations (8,643 unique patients) were included. AIR, CDR, and CDR adjusted for pathology confirmation rate were 45.4%, 23.8%, and 27.6% for center I; 47.2%, 20.0%, and 22.8% for center II; and 42.3%, 27.2%, and 30.1% for center III, respectively. Pathology confirmation rate ranged from 81.6% to 88.0% across three centers. AIR and CDR for biopsy-naive patients were 45.5% to 52.6% and 24.2% to 33.5% across three centers, respectively, and those for previous negative biopsy were 27.2% to 39.8% and 11.7% to 14.2% across three centers, respectively. CONCLUSION: We reported CDR and AIR in prostate MRI for clinical suspicion of PCa. CDR needs to be adjusted for pathology confirmation rate and pre-MRI biopsy status for interfacility comparison.

Integrating Electronic Health Record Data into the ADEpedia-on-OHDSI Platform for Improved Signal Detection: A Case Study of Immune-related Adverse Events.

With widespread adoption of electronic health records (EHRs), Real World Data and Real World Evidence (RWE) have been increasingly used by FDA for evaluating drug safety and effectiveness. However, integration of heterogeneous drug safety data sources and systems remains an impediment for effective pharmacovigilance studies. In an ongoing project, we have developed a next generation pharmacovigilance signal detection framework known as ADEpedia-on-OHDSI using the OMOP common data model (CDM). The objective of the study is to demonstrate the feasibility of the framework for integrating both spontaneous reporting data and EHR data for improved signal detection with a case study of immune-related adverse events. We first loaded the OMOP CDM with both recent and legacy FAERS (FDA Adverse Event Reporting System) data (from the time period between Jan. 2004 and Dec. 2018). We also integrated the clinical data from the Mayo Clinic EHR system for six oncological immunotherapy drugs. We implemented a signal detection algorithm and compared the timelines of positive signals detected from both FAERS and EHR data. We found that the signals detected from EHRs are 4 months earlier than signals detected from FAERS database (depending on the signal detection methods used) for the ipilimumab-induced hypopituitarism. Our CDM-based approach would be useful to provide a scalable solution to integrate both drug safety data and EHR data to generate RWE for improved signal detection.

Detecting and Filtering Immune-Related Adverse Events Signal Based on Text Mining and Observational Health Data Sciences and Informatics Common Data Model: Framework Development Study.

BACKGROUND: Immune checkpoint inhibitors are associated with unique immune-related adverse events (irAEs). As most of the immune checkpoint inhibitors are new to the market, it is important to conduct studies using real-world data sources to investigate their safety profiles. OBJECTIVE: The aim of the study was to develop a framework for signal detection and filtration of novel irAEs for 6 Food and Drug Administration-approved immune checkpoint inhibitors. METHODS: In our framework, we first used the Food and Drug Administration's Adverse Event Reporting System (FAERS) standardized in an Observational Health Data Sciences and Informatics (OHDSI) common data model (CDM) to collect immune checkpoint inhibitor-related event data and conducted irAE signal detection. OHDSI CDM is a standard-driven data model that focuses on transforming different databases into a common format and standardizing medical terms to a common representation. We then filtered those already known irAEs from drug labels and literature by using a customized text-mining pipeline based on clinical text analysis and knowledge extraction system with Medical Dictionary for Regulatory Activities (MedDRA) as a dictionary. Finally, we classified the irAE detection results into three different categories to discover potentially new irAE signals. RESULTS: By our text-mining pipeline, 490 irAE terms were identified from drug labels, and 918 terms were identified from the literature. In addition, of the 94 positive signals detected using CDM-based FAERS, 53 signals (56%) were labeled signals, 10 (11%) were unlabeled published signals, and 31 (33%) were potentially new signals. CONCLUSIONS: We demonstrated that our approach is effective for irAE signal detection and filtration. Moreover, our CDM-based framework could facilitate adverse drug events detection and filtration toward the goal of next-generation pharmacovigilance that seamlessly integrates electronic health record data for improved signal detection.

Coverage Evaluation of CTCAE for Capturing the Immune-related Adverse Events Leveraging Text Mining Technologies.

Background: Immunotherapy is seen as a promising new treatment for cancer but it may also cause immune-related adverse events (irAEs). Post-market surveillance of immunotherapy drugs highly depends on the ability to capture and standardize irAE data. The Common Terminology Criteria for Adverse Events (CTCAE) is a potential terminology that can be leveraged for irAEs standardization. However, the capability of the CTCAE in irAEs standardization needs to be evaluated. Methods: We investigated the irAEs of six FDA approved cancer immunotherapy monoclonal antibodies (mAbs) and evaluated the coverage of the CTCAE for capturing irAEs. We manually identified irAEs from drug labels of the 6 mAbs as the gold standard. We assessed the performance of two text mining pipelines using the dictionary lookup of the CTCAE terms and identified irAEs. In the coverage evaluation, the CTCAE was compared with MedDRA, a standard terminology for regulatory science, for irAE standardization. Results: We manually identified 510 unique irAEs from the drug labels. When using the CTCAE as a dictionary to run the text mining pipeline, the precision, recall and F-measure value was 100%, 10.78% and 19.47%. After adding manually identified irAE terms into the dictionary, the recall and F-measure value significantly improved, increased to 95.69% and 97.31%, respectively. In the coverage evaluation, compared with MedDRA, the coverage rate of the CTCAE is only 13.50% when taking all the mining results together into consideration. Conclusion: With some limitations in our study, we clearly demonstrated that the CTCAE needs an extension to meet the irAE standardization task.