Artificial intelligence-powered pharmacovigilance: A review of machine and deep learning in clinical text-based adverse drug event detection for benchmark datasets.

OBJECTIVE: The primary objective of this review is to investigate the effectiveness of machine learning and deep learning methodologies in the context of extracting adverse drug events (ADEs) from clinical benchmark datasets. We conduct an in-depth analysis, aiming to compare the merits and drawbacks of both machine learning and deep learning techniques, particularly within the framework of named-entity recognition (NER) and relation classification (RC) tasks related to ADE extraction. Additionally, our focus extends to the examination of specific features and their impact on the overall performance of these methodologies. In a broader perspective, our research extends to ADE extraction from various sources, including biomedical literature, social media data, and drug labels, removing the limitation to exclusively machine learning or deep learning methods. METHODS: We conducted an extensive literature review on PubMed using the query "(((machine learning [Medical Subject Headings (MeSH) Terms]) OR (deep learning [MeSH Terms])) AND (adverse drug event [MeSH Terms])) AND (extraction)", and supplemented this with a snowballing approach to review 275 references sourced from retrieved articles. RESULTS: In our analysis, we included twelve articles for review. For the NER task, deep learning models outperformed machine learning models. In the RC task, gradient Boosting, multilayer perceptron and random forest models excelled. The Bidirectional Encoder Representations from Transformers (BERT) model consistently achieved the best performance in the end-to-end task. Future efforts in the end-to-end task should prioritize improving NER accuracy, especially for 'ADE' and 'Reason'. CONCLUSION: These findings hold significant implications for advancing the field of ADE extraction and pharmacovigilance, ultimately contributing to improved drug safety monitoring and healthcare outcomes.

Measuring and controlling medical record abstraction (MRA) error rates in an observational study.

BACKGROUND: Studies have shown that data collection by medical record abstraction (MRA) is a significant source of error in clinical research studies relying on secondary use data. Yet, the quality of data collected using MRA is seldom assessed. We employed a novel, theory-based framework for data quality assurance and quality control of MRA. The objective of this work is to determine the potential impact of formalized MRA training and continuous quality control (QC) processes on data quality over time. METHODS: We conducted a retrospective analysis of QC data collected during a cross-sectional medical record review of mother-infant dyads with Neonatal Opioid Withdrawal Syndrome. A confidence interval approach was used to calculate crude (Wald's method) and adjusted (generalized estimating equation) error rates over time. We calculated error rates using the number of errors divided by total fields ("all-field" error rate) and populated fields ("populated-field" error rate) as the denominators, to provide both an optimistic and a conservative measurement, respectively. RESULTS: On average, the ACT NOW CE Study maintained an error rate between 1% (optimistic) and 3% (conservative). Additionally, we observed a decrease of 0.51 percentage points with each additional QC Event conducted. CONCLUSIONS: Formalized MRA training and continuous QC resulted in lower error rates than have been found in previous literature and a decrease in error rates over time. This study newly demonstrates the importance of continuous process controls for MRA within the context of a multi-site clinical research study.

Exploring the implications of the new ICD-10-CM classification system for injury surveillance: analysis of dually coded data from two medical centres.

INTRODUCTION: External cause of injury matrices is used to classify mechanisms/causes of injuries for surveillance and research. Little is known about the performance of the Centers for Disease Control and Prevention's new external cause of injury matrix for Clinical Modification of the 10th Revision of the International Classification of Diseases (ICD-10-CM), compared with the ICD-9-CM version. METHODS: Dually coded (ICD-9-CM and ICD-10-CM) administrative data were obtained from two major academic trauma centres. Injury-related cases were identified and categorised by mechanism/cause and manner/intent. Comparability ratios (CR) were used to estimate the net impact of changing from ICD-9-CM to ICD-10-CM on the number of cases classified to each mechanism/cause category. Chamberlain's percent positive agreements (PPA) were calculated and McNemar's test was used to assess the significance of observed classification differences. RESULTS: Of 4832 and 5211 dual-coded records from the two centres, 632 and 520 with injury-related principal diagnoses and external cause codes in both ICD-9-CM and ICD-10-CM were identified. CRs for the mechanisms/causes with at least 20 records ranged from 0.85 to 1.9 at one centre and from 0.97 to 1.07 at the other. Among these mechanisms/causes, PPAs ranged from 33% for 'other transport' to 94% for poisoning at one centre, and from 75% for 'other transport' to 100% for fires/burns at the other centre. Case assignment differed significantly for falls, motor vehicle traffic, other transport, and 'struck by/against' injuries at one centre, and for 'other pedal cyclist' at the other centre. CONCLUSION: Switching to ICD-10-CM and the new external cause of injury matrix may affect injury surveillance and research, especially for certain mechanisms/causes.

Building a More Diverse Public Health Informatics Workforce: Preliminary Results.

The US public health infrastructure has been historically underfunded, a condition that was exacerbated by the COVID-19 pandemic. This was especially noted in the area of public health informatics. It was also acknowledged that the lack of a diverse public health workforce made it more difficult to address biases and disparities effectively. In 2021 the Office of the National Coordinator awarded $73 million to 10 awardees to develop public health informatics and technology (PHIT) workforce training. The Gaining Equity in Training for Public Health Informatics and Technology (GET PHIT) award utilizes various methods to train and engage minority and underserved populations in the field of public health informatics. Evaluations of the bootcamps and internships to date have shown generally positive results, both in terms of skills acquired and overall experiences. These results indicate that integrating the fields of public health and data science in non-degree, short-term experiences can have positive outcomes.

An ICD for the Digital World: What Does the ICD-11 Research Show?

The 11th revision of the International Classification of Diseases (ICD) is now available for use. A literature search was conducted to review and summarize the research conducted to date. In addition to the ease of integration into electronic health records using standard digital tools such as uniform resource identifiers and application programming interfaces, ICD-11 and the World Health Organization provided linearization for mortality and morbidity, ICD-11-MMS, promise improved backward compatibility to ICD-10; increased availability in multiple languages; greater detail for clinical use, including traditional Chinese medicine; and enhanced maintenance for continued relevance. The studies reviewed here support the superior content and utility of ICD-11-MMS. Meaningful planning for implementation has begun, including the provision of a framework. It is time for the world to adopt a digitally prepared ICD.

Insights on the Future of Digital Health: An Analysis of Twitter Posts of IMIA Fellows.

The COVID-19 pandemic has reshaped technology-enhanced services in health and care organizations globally. As the world pivots towards a post-COVID-19 environment, it is essential to examine emerging trends amongst thought leaders in the health information technology sector. This study queried Twitter feeds of IMIA Fellows from 2013 through 2022, utilizing combinations of sentiment analysis, latent dirichlet allocation, and document analysis methods. The results provided a glimpse of positive sentiment year upon year, with the most negative sentiment prevalent in 2020, due to the onset of the pandemic. The findings from this study can be strategically used to analyze emerging trends in digital health, as well as to shape health IT thought leadership in the post-pandemic landscape.

Aligning the American Health Information Management Association Entry-level Curricula Competencies and Career Map With Industry Job Postings: Cross-sectional Study.

BACKGROUND: The field of health information management (HIM) focuses on the protection and management of health information from a variety of sources. The American Health Information Management Association (AHIMA) Council for Excellence in Education (CEE) determines the needed skills and competencies for this field. AHIMA's HIM curricula competencies are divided into several domains among the associate, undergraduate, and graduate levels. Moreover, AHIMA's career map displays career paths for HIM professionals. What is not known is whether these competencies and the career map align with industry demands. OBJECTIVE: The primary aim of this study is to analyze HIM job postings on a US national job recruiting website to determine whether the job postings align with recognized HIM domains, while the secondary aim is to evaluate the AHIMA career map to determine whether it aligns with the job postings. METHODS: A national job recruitment website was mined electronically (web scraping) using the search term "health information management." This cross-sectional inquiry evaluated job advertisements during a 2-week period in 2021. After the exclusion criteria, 691 job postings were analyzed. Data were evaluated with descriptive statistics and natural language processing (NLP). Soft cosine measures (SCM) were used to determine correlations between job postings and the AHIMA career map, curricular competencies, and curricular considerations. ANOVA was used to determine statistical significance. RESULTS: Of all the job postings, 29% (140/691) were in the Southeast, followed by the Midwest (140/691, 20%), West (131/691,19%), Northeast (94/691, 14%), and Southwest (73/691, 11%). The educational levels requested were evenly distributed between high school diploma (219/691, 31.7%), associate degree (269/691, 38.6%), or bachelor's degree (225/691, 32.5%). A master's degree was requested in only 8% (52/691) of the postings, with 72% (42/58) preferring one and 28% (16/58) requiring one. A Registered Health Information Technologist (RHIT) credential was the most commonly requested (207/691, 29.9%) in job postings, followed by Registered Health Information Administrator (RHIA; 180/691, 26%) credential. SCM scores were significantly higher in the informatics category compared to the coding and revenue cycle (P=.006) and data analytics categories (P<.001) but not significantly different from the information governance category (P=.85). The coding and revenue cycle category had a significantly higher SCM score compared to the data analytics category (P<.001). Additionally, the information governance category was significantly higher than the data analytics category (P<.001). SCM scores were significantly different between each competency category, except there were no differences in the average SCM score between the information protection and revenue cycle management categories (P=.96) and the information protection and data structure, content, and information governance categories (P=.31). CONCLUSIONS: Industry job postings primarily sought degrees, with a master's degree a distant fourth. NLP analysis of job postings suggested that the correlation between the informatics category and job postings was higher than that of the coding, revenue cycle, and data analytics categories.

Preliminary study of patient safety and quality use cases for ICD-11 MMS.

OBJECTIVE: This study investigated how well-suited the International Classification of Diseases, 11th Revision, for Mortality and Morbidity Statistics, (ICD-11 MMS) is for 2 morbidity use cases, patient safety and quality, examining the level of detail captured, and evaluating the necessity for the development of a US clinical modification (CM). MATERIALS AND METHODS: Utilizing the 5 NCVHS-specified perspectives plus the consumer perspective, a framework was created of International Classification of Diseases, 10th Revision, Clinical Modification (ICD-10-CM) use cases. Analysis yielded candidate source criteria for use in case evaluation. Patient safety and quality were chosen because they are relevant across all perspectives.Granularity differences and content coverage of ICD-11 MMS entities were assessed pre- and post-coordination to determine suitability for the 2 use cases. Pressure ulcers, a common condition across 3 patient safety applications, became the focus for comparing ICD-10-CM codes to ICD-11 MMS codes. For 3 electronic clinical quality measures (eCQMs), the evaluation centered on specified value sets for ischemic stroke, hypertension, and diabetes. RESULTS: For pressure ulcers, the ICD-11 MMS was found to exceed ICD-10-CM capabilities via post-coordinated extension codes. For the 3 eCQM value sets explored, the ICD-11 MMS fully represented the disease concepts when post-coordinated code clusters were used. CONCLUSIONS: The examples from the patient safety and quality use cases evaluated in this study are appropriate for ICD-11 MMS. It captures greater detail than ICD-10-CM, and ICD-11 MMS specificity would benefit both use cases. The authors believe this preliminary study indicates the US should invest resources to explore adopting the WHO ICD-11 MMS and tooling and guidelines to implement post-coordination.

Lessons Learned for Identifying and Annotating Permissions in Clinical Consent Forms.

BACKGROUND: The lack of machine-interpretable representations of consent permissions precludes development of tools that act upon permissions across information ecosystems, at scale. OBJECTIVES: To report the process, results, and lessons learned while annotating permissions in clinical consent forms. METHODS: We conducted a retrospective analysis of clinical consent forms. We developed an annotation scheme following the MAMA (Model-Annotate-Model-Annotate) cycle and evaluated interannotator agreement (IAA) using observed agreement (A o), weighted kappa (κw ), and Krippendorff's α. RESULTS: The final dataset included 6,399 sentences from 134 clinical consent forms. Complete agreement was achieved for 5,871 sentences, including 211 positively identified and 5,660 negatively identified as permission-sentences across all three annotators (A o = 0.944, Krippendorff's α = 0.599). These values reflect moderate to substantial IAA. Although permission-sentences contain a set of common words and structure, disagreements between annotators are largely explained by lexical variability and ambiguity in sentence meaning. CONCLUSION: Our findings point to the complexity of identifying permission-sentences within the clinical consent forms. We present our results in light of lessons learned, which may serve as a launching point for developing tools for automated permission extraction.

Evaluating the Coverage of the HL7 ® FHIR ® Standard to Support eSource Data Exchange Implementations for use in Multi-Site Clinical Research Studies.

The direct use of EHR data in research, often referred to as 'eSource', has long-been a goal for researchers because of anticipated increases in data quality and reductions in site burden. eSource solutions should rely on data exchange standards for consistency, quality, and efficiency. The utility of any data standard can be evaluated by its ability to meet specific use case requirements. The Health Level Seven (HL7 ® ) Fast Healthcare Interoperability Resources (FHIR ® ) standard is widely recognized for clinical data exchange; however, a thorough analysis of the standard's data coverage in supporting multi-site clinical studies has not been conducted. We developed and implemented a systematic mapping approach for evaluating HL7 ® FHIR ® standard coverage in multi-center clinical trials. Study data elements from three diverse studies were mapped to HL7 ® FHIR ® resources, offering insight into the coverage and utility of the standard for supporting the data collection needs of multi-site clinical research studies.

Training Leaders in Health Informatics.

Despite the widespread adoption of electronic health records (EHRs) in the U.S. over the past decade, significant improvements, especially in patient safety, have yet to be realized. This finding, along with health informatics workforce data and an identified gap in the offerings of an educational program, led to a proposed professional doctorate in health informatics. Developed via stakeholder focus groups, the program was approved by the public university system, the state-level educational authority, and the regional accreditation body, with final approval in July 2018. Unique features of the program include a prolonged practice project demonstrating a return on investment, as well as online and face-to-face delivery components. This program aims to develop evidence-based professionals who improve the health of people and populations through the application of health informatics. Applications and interest in the first class are high.

Evidence-based Operations Management in Health Information Management: A Case Study.

This is a case study of the evidence-based management practices of a centralized health information management (HIM) department in a large integrated healthcare delivery system. The case study used interviews and focus groups, as well as de-identified dashboards, to explore the impact of reporting on the organization. The dashboards and key performance indicators (KPIs) were initially developed in 2012 and have continued to evolve. The themes that resulted include the following: (1) evidence-based management is integral to the culture of the organization; (2) communicating regularly via dashboards and KPIs is key to transmitting the value of HIM to the entire organization; and (3) staff not only report the required measures for the dashboard but also take pride in it and often develop methods for tracking their individual performance. Most evidence supporting HIM operations management is related to coding and clinical documentation improvement, but even in those areas, national benchmarks are missing. It is important for the HIM profession to develop national and regional benchmarks to assist professionals in managing operations effectively and communicating their value to the healthcare industry.

Global Workforce Trends in Health Informatics & Information Management.

In a data driven environment, healthcare has seen ongoing digital transformation to meet both clinical and business needs. But, have the educational and functional requirements of the health informatics and information management (HIIM) workforce also adapted? This study examined the current employment opportunities in HIIM globally. Using 11 keywords generated from a literature review, postings on the job advertisement website Indeed™ for all available countries were analyzed. The results show that job postings tend to fall within 4 discrete categories: 1) health information technology; 2) health research; 3) health leadership and project management; and 4) health compliance. Data indicated a higher prevalence for certain areas by country. The findings from this study can inform HIIM educational providers about future skill requirements.

An Exploratory Study Demonstrating the Health Information Management Profession as a STEM Discipline.

It is time to make the case for health information management (HIM) to be included in science, technology, engineering, and math (STEM) education. A careful review of the HIM competencies approved by the American Health Information Management Association (AHIMA) illustrates the role of HIM professionals in informatics, data analytics, and data use. More precisely, the competency subdomains clearly align with content in the STEM disciplines of science, math, and technology, and the individual competencies or tasks in each subdomain solidify the assertion that HIM should be considered part of the STEM disciplines. Evaluation of AHIMA membership data showed that, at the education and work setting levels, AHIMA members are employed in many areas that are common to both HIM and STEM.

EHR's effect on the revenue cycle management Coding function.

Without administrative terminologies there is no revenue to manage. The use of healthcare IT to capture the codes for administrative and financial support functions will impact the revenue cycle and the management of it. This is presumed to occur because clinical data coded at the point of care becomes the source for claims data. Thus, as electronic health record system applications utilizing terminologies are implemented, healthcare providers need to systematically consider the effect on the coding function and management of the revenue cycle. A key factor is the sequence of events changes, i.e., instead of a health information management professional selecting billing codes at the conclusion of an encounter based on the review of the record, clinical data generates the claims data via mapping. Efficiencies and management challenges result.

A comparison of primary care information content in UpToDate and the National Guideline Clearinghouse.

OBJECTIVES: The study sought to determine if two major resources for primary care questions have significant differences in information content and whether the number of documents found differs by disease category, patient age, or patient gender. METHODS: Seven hundred fifty-two questions were randomly selected from the Clinical Questions Collection of the National Library of Medicine. UpToDate and the National Guidelines Clearinghouse (NGC) were searched utilizing keywords from the questions. The number of documents retrieved for each question in the resources was recorded. Chi-squared analysis was used to compare differences in retrieval between the resources. Logistic regression was used to evaluate the effect of patient age, patient gender, or disease category on the ability to find content. RESULTS: UpToDate returned 1 or more documents for 580 questions, while NGC returned at least 1 document for 493 questions (77.1% versus 65.5% of question sampled, P = 0.001). In combination, the 2 resources returned content for 91% of searches (n = 685). NGC retrieved a mean of 16.3 documents per question versus 8.7 documents from UpToDate. Disease category was the only variable having a significant impact on the presence of online resource content. CONCLUSIONS: UpToDate had greater breadth of content than NGC, while neither resource provided complete coverage. Current practice guidelines, as reflected by those in the NGC, addressed at most two-thirds of the selected clinical questions.