Measuring and controlling medical record abstraction (MRA) error rates in an observational study.

BACKGROUND: Studies have shown that data collection by medical record abstraction (MRA) is a significant source of error in clinical research studies relying on secondary use data. Yet, the quality of data collected using MRA is seldom assessed. We employed a novel, theory-based framework for data quality assurance and quality control of MRA. The objective of this work is to determine the potential impact of formalized MRA training and continuous quality control (QC) processes on data quality over time. METHODS: We conducted a retrospective analysis of QC data collected during a cross-sectional medical record review of mother-infant dyads with Neonatal Opioid Withdrawal Syndrome. A confidence interval approach was used to calculate crude (Wald's method) and adjusted (generalized estimating equation) error rates over time. We calculated error rates using the number of errors divided by total fields ("all-field" error rate) and populated fields ("populated-field" error rate) as the denominators, to provide both an optimistic and a conservative measurement, respectively. RESULTS: On average, the ACT NOW CE Study maintained an error rate between 1% (optimistic) and 3% (conservative). Additionally, we observed a decrease of 0.51 percentage points with each additional QC Event conducted. CONCLUSIONS: Formalized MRA training and continuous QC resulted in lower error rates than have been found in previous literature and a decrease in error rates over time. This study newly demonstrates the importance of continuous process controls for MRA within the context of a multi-site clinical research study.

Exploring the implications of the new ICD-10-CM classification system for injury surveillance: analysis of dually coded data from two medical centres.

INTRODUCTION: External cause of injury matrices is used to classify mechanisms/causes of injuries for surveillance and research. Little is known about the performance of the Centers for Disease Control and Prevention's new external cause of injury matrix for Clinical Modification of the 10th Revision of the International Classification of Diseases (ICD-10-CM), compared with the ICD-9-CM version. METHODS: Dually coded (ICD-9-CM and ICD-10-CM) administrative data were obtained from two major academic trauma centres. Injury-related cases were identified and categorised by mechanism/cause and manner/intent. Comparability ratios (CR) were used to estimate the net impact of changing from ICD-9-CM to ICD-10-CM on the number of cases classified to each mechanism/cause category. Chamberlain's percent positive agreements (PPA) were calculated and McNemar's test was used to assess the significance of observed classification differences. RESULTS: Of 4832 and 5211 dual-coded records from the two centres, 632 and 520 with injury-related principal diagnoses and external cause codes in both ICD-9-CM and ICD-10-CM were identified. CRs for the mechanisms/causes with at least 20 records ranged from 0.85 to 1.9 at one centre and from 0.97 to 1.07 at the other. Among these mechanisms/causes, PPAs ranged from 33% for 'other transport' to 94% for poisoning at one centre, and from 75% for 'other transport' to 100% for fires/burns at the other centre. Case assignment differed significantly for falls, motor vehicle traffic, other transport, and 'struck by/against' injuries at one centre, and for 'other pedal cyclist' at the other centre. CONCLUSION: Switching to ICD-10-CM and the new external cause of injury matrix may affect injury surveillance and research, especially for certain mechanisms/causes.

Building a More Diverse Public Health Informatics Workforce: Preliminary Results.

The US public health infrastructure has been historically underfunded, a condition that was exacerbated by the COVID-19 pandemic. This was especially noted in the area of public health informatics. It was also acknowledged that the lack of a diverse public health workforce made it more difficult to address biases and disparities effectively. In 2021 the Office of the National Coordinator awarded $73 million to 10 awardees to develop public health informatics and technology (PHIT) workforce training. The Gaining Equity in Training for Public Health Informatics and Technology (GET PHIT) award utilizes various methods to train and engage minority and underserved populations in the field of public health informatics. Evaluations of the bootcamps and internships to date have shown generally positive results, both in terms of skills acquired and overall experiences. These results indicate that integrating the fields of public health and data science in non-degree, short-term experiences can have positive outcomes.

An ICD for the Digital World: What Does the ICD-11 Research Show?

The 11th revision of the International Classification of Diseases (ICD) is now available for use. A literature search was conducted to review and summarize the research conducted to date. In addition to the ease of integration into electronic health records using standard digital tools such as uniform resource identifiers and application programming interfaces, ICD-11 and the World Health Organization provided linearization for mortality and morbidity, ICD-11-MMS, promise improved backward compatibility to ICD-10; increased availability in multiple languages; greater detail for clinical use, including traditional Chinese medicine; and enhanced maintenance for continued relevance. The studies reviewed here support the superior content and utility of ICD-11-MMS. Meaningful planning for implementation has begun, including the provision of a framework. It is time for the world to adopt a digitally prepared ICD.

Insights on the Future of Digital Health: An Analysis of Twitter Posts of IMIA Fellows.

The COVID-19 pandemic has reshaped technology-enhanced services in health and care organizations globally. As the world pivots towards a post-COVID-19 environment, it is essential to examine emerging trends amongst thought leaders in the health information technology sector. This study queried Twitter feeds of IMIA Fellows from 2013 through 2022, utilizing combinations of sentiment analysis, latent dirichlet allocation, and document analysis methods. The results provided a glimpse of positive sentiment year upon year, with the most negative sentiment prevalent in 2020, due to the onset of the pandemic. The findings from this study can be strategically used to analyze emerging trends in digital health, as well as to shape health IT thought leadership in the post-pandemic landscape.

Aligning the American Health Information Management Association Entry-level Curricula Competencies and Career Map With Industry Job Postings: Cross-sectional Study.

BACKGROUND: The field of health information management (HIM) focuses on the protection and management of health information from a variety of sources. The American Health Information Management Association (AHIMA) Council for Excellence in Education (CEE) determines the needed skills and competencies for this field. AHIMA's HIM curricula competencies are divided into several domains among the associate, undergraduate, and graduate levels. Moreover, AHIMA's career map displays career paths for HIM professionals. What is not known is whether these competencies and the career map align with industry demands. OBJECTIVE: The primary aim of this study is to analyze HIM job postings on a US national job recruiting website to determine whether the job postings align with recognized HIM domains, while the secondary aim is to evaluate the AHIMA career map to determine whether it aligns with the job postings. METHODS: A national job recruitment website was mined electronically (web scraping) using the search term "health information management." This cross-sectional inquiry evaluated job advertisements during a 2-week period in 2021. After the exclusion criteria, 691 job postings were analyzed. Data were evaluated with descriptive statistics and natural language processing (NLP). Soft cosine measures (SCM) were used to determine correlations between job postings and the AHIMA career map, curricular competencies, and curricular considerations. ANOVA was used to determine statistical significance. RESULTS: Of all the job postings, 29% (140/691) were in the Southeast, followed by the Midwest (140/691, 20%), West (131/691,19%), Northeast (94/691, 14%), and Southwest (73/691, 11%). The educational levels requested were evenly distributed between high school diploma (219/691, 31.7%), associate degree (269/691, 38.6%), or bachelor's degree (225/691, 32.5%). A master's degree was requested in only 8% (52/691) of the postings, with 72% (42/58) preferring one and 28% (16/58) requiring one. A Registered Health Information Technologist (RHIT) credential was the most commonly requested (207/691, 29.9%) in job postings, followed by Registered Health Information Administrator (RHIA; 180/691, 26%) credential. SCM scores were significantly higher in the informatics category compared to the coding and revenue cycle (P=.006) and data analytics categories (P<.001) but not significantly different from the information governance category (P=.85). The coding and revenue cycle category had a significantly higher SCM score compared to the data analytics category (P<.001). Additionally, the information governance category was significantly higher than the data analytics category (P<.001). SCM scores were significantly different between each competency category, except there were no differences in the average SCM score between the information protection and revenue cycle management categories (P=.96) and the information protection and data structure, content, and information governance categories (P=.31). CONCLUSIONS: Industry job postings primarily sought degrees, with a master's degree a distant fourth. NLP analysis of job postings suggested that the correlation between the informatics category and job postings was higher than that of the coding, revenue cycle, and data analytics categories.

Preliminary study of patient safety and quality use cases for ICD-11 MMS.

OBJECTIVE: This study investigated how well-suited the International Classification of Diseases, 11th Revision, for Mortality and Morbidity Statistics, (ICD-11 MMS) is for 2 morbidity use cases, patient safety and quality, examining the level of detail captured, and evaluating the necessity for the development of a US clinical modification (CM). MATERIALS AND METHODS: Utilizing the 5 NCVHS-specified perspectives plus the consumer perspective, a framework was created of International Classification of Diseases, 10th Revision, Clinical Modification (ICD-10-CM) use cases. Analysis yielded candidate source criteria for use in case evaluation. Patient safety and quality were chosen because they are relevant across all perspectives.Granularity differences and content coverage of ICD-11 MMS entities were assessed pre- and post-coordination to determine suitability for the 2 use cases. Pressure ulcers, a common condition across 3 patient safety applications, became the focus for comparing ICD-10-CM codes to ICD-11 MMS codes. For 3 electronic clinical quality measures (eCQMs), the evaluation centered on specified value sets for ischemic stroke, hypertension, and diabetes. RESULTS: For pressure ulcers, the ICD-11 MMS was found to exceed ICD-10-CM capabilities via post-coordinated extension codes. For the 3 eCQM value sets explored, the ICD-11 MMS fully represented the disease concepts when post-coordinated code clusters were used. CONCLUSIONS: The examples from the patient safety and quality use cases evaluated in this study are appropriate for ICD-11 MMS. It captures greater detail than ICD-10-CM, and ICD-11 MMS specificity would benefit both use cases. The authors believe this preliminary study indicates the US should invest resources to explore adopting the WHO ICD-11 MMS and tooling and guidelines to implement post-coordination.

Lessons Learned for Identifying and Annotating Permissions in Clinical Consent Forms.

BACKGROUND: The lack of machine-interpretable representations of consent permissions precludes development of tools that act upon permissions across information ecosystems, at scale. OBJECTIVES: To report the process, results, and lessons learned while annotating permissions in clinical consent forms. METHODS: We conducted a retrospective analysis of clinical consent forms. We developed an annotation scheme following the MAMA (Model-Annotate-Model-Annotate) cycle and evaluated interannotator agreement (IAA) using observed agreement (A o), weighted kappa (κw ), and Krippendorff's α. RESULTS: The final dataset included 6,399 sentences from 134 clinical consent forms. Complete agreement was achieved for 5,871 sentences, including 211 positively identified and 5,660 negatively identified as permission-sentences across all three annotators (A o = 0.944, Krippendorff's α = 0.599). These values reflect moderate to substantial IAA. Although permission-sentences contain a set of common words and structure, disagreements between annotators are largely explained by lexical variability and ambiguity in sentence meaning. CONCLUSION: Our findings point to the complexity of identifying permission-sentences within the clinical consent forms. We present our results in light of lessons learned, which may serve as a launching point for developing tools for automated permission extraction.

Evidence-based Operations Management in Health Information Management: A Case Study.

This is a case study of the evidence-based management practices of a centralized health information management (HIM) department in a large integrated healthcare delivery system. The case study used interviews and focus groups, as well as de-identified dashboards, to explore the impact of reporting on the organization. The dashboards and key performance indicators (KPIs) were initially developed in 2012 and have continued to evolve. The themes that resulted include the following: (1) evidence-based management is integral to the culture of the organization; (2) communicating regularly via dashboards and KPIs is key to transmitting the value of HIM to the entire organization; and (3) staff not only report the required measures for the dashboard but also take pride in it and often develop methods for tracking their individual performance. Most evidence supporting HIM operations management is related to coding and clinical documentation improvement, but even in those areas, national benchmarks are missing. It is important for the HIM profession to develop national and regional benchmarks to assist professionals in managing operations effectively and communicating their value to the healthcare industry.

Training Leaders in Health Informatics.

Despite the widespread adoption of electronic health records (EHRs) in the U.S. over the past decade, significant improvements, especially in patient safety, have yet to be realized. This finding, along with health informatics workforce data and an identified gap in the offerings of an educational program, led to a proposed professional doctorate in health informatics. Developed via stakeholder focus groups, the program was approved by the public university system, the state-level educational authority, and the regional accreditation body, with final approval in July 2018. Unique features of the program include a prolonged practice project demonstrating a return on investment, as well as online and face-to-face delivery components. This program aims to develop evidence-based professionals who improve the health of people and populations through the application of health informatics. Applications and interest in the first class are high.

Global Workforce Trends in Health Informatics & Information Management.

In a data driven environment, healthcare has seen ongoing digital transformation to meet both clinical and business needs. But, have the educational and functional requirements of the health informatics and information management (HIIM) workforce also adapted? This study examined the current employment opportunities in HIIM globally. Using 11 keywords generated from a literature review, postings on the job advertisement website Indeed™ for all available countries were analyzed. The results show that job postings tend to fall within 4 discrete categories: 1) health information technology; 2) health research; 3) health leadership and project management; and 4) health compliance. Data indicated a higher prevalence for certain areas by country. The findings from this study can inform HIIM educational providers about future skill requirements.

An Exploratory Study Demonstrating the Health Information Management Profession as a STEM Discipline.

It is time to make the case for health information management (HIM) to be included in science, technology, engineering, and math (STEM) education. A careful review of the HIM competencies approved by the American Health Information Management Association (AHIMA) illustrates the role of HIM professionals in informatics, data analytics, and data use. More precisely, the competency subdomains clearly align with content in the STEM disciplines of science, math, and technology, and the individual competencies or tasks in each subdomain solidify the assertion that HIM should be considered part of the STEM disciplines. Evaluation of AHIMA membership data showed that, at the education and work setting levels, AHIMA members are employed in many areas that are common to both HIM and STEM.

A comparison of primary care information content in UpToDate and the National Guideline Clearinghouse.

OBJECTIVES: The study sought to determine if two major resources for primary care questions have significant differences in information content and whether the number of documents found differs by disease category, patient age, or patient gender. METHODS: Seven hundred fifty-two questions were randomly selected from the Clinical Questions Collection of the National Library of Medicine. UpToDate and the National Guidelines Clearinghouse (NGC) were searched utilizing keywords from the questions. The number of documents retrieved for each question in the resources was recorded. Chi-squared analysis was used to compare differences in retrieval between the resources. Logistic regression was used to evaluate the effect of patient age, patient gender, or disease category on the ability to find content. RESULTS: UpToDate returned 1 or more documents for 580 questions, while NGC returned at least 1 document for 493 questions (77.1% versus 65.5% of question sampled, P = 0.001). In combination, the 2 resources returned content for 91% of searches (n = 685). NGC retrieved a mean of 16.3 documents per question versus 8.7 documents from UpToDate. Disease category was the only variable having a significant impact on the presence of online resource content. CONCLUSIONS: UpToDate had greater breadth of content than NGC, while neither resource provided complete coverage. Current practice guidelines, as reflected by those in the NGC, addressed at most two-thirds of the selected clinical questions.

Results from the Veterans Health Administration ICD-10-CM/PCS Coding Pilot Study.

The Veterans Health Administration (VHA) of the US Department of Veterans Affairs has been preparing for the October 1, 2015, conversion to the International Classification of Diseases, Tenth Revision, Clinical Modification and Procedural Coding System (ICD-10-CM/PCS) for more than four years. The VHA's Office of Informatics and Analytics ICD-10 Program Management Office established an ICD-10 Learning Lab to explore expected operational challenges. This study was conducted to determine the effects of the classification system conversion on coding productivity. ICD codes are integral to VHA business processes and are used for purposes such as clinical studies, performance measurement, workload capture, cost determination, Veterans Equitable Resource Allocation (VERA) determination, morbidity and mortality classification, indexing of hospital records by disease and operations, data storage and retrieval, research purposes, and reimbursement. The data collection for this study occurred in multiple VHA sites across several months using standardized methods. It is commonly accepted that coding productivity will decrease with the implementation of ICD-10-CM/PCS. The findings of this study suggest that the decrease will be more significant for inpatient coding productivity (64.5 percent productivity decrease) than for ambulatory care coding productivity (6.7 percent productivity decrease). This study reveals the following important points regarding ICD-10-CM/PCS coding productivity: 1. Ambulatory care ICD-10-CM coding productivity is not expected to decrease as significantly as inpatient ICD-10-CM/PCS coding productivity. 2. Coder training and type of record (inpatient versus outpatient) affect coding productivity. 3. Inpatient coding productivity is decreased when a procedure requiring ICD-10-PCS coding is present. It is highly recommended that organizations perform their own analyses to determine the effects of ICD-10-CM/PCS implementation on coding productivity.

Health information technology workforce needs of rural primary care practices.

PURPOSE: This study assessed electronic health record (EHR) and health information technology (HIT) workforce resources needed by rural primary care practices, and their workforce-related barriers to implementing and using EHRs and HIT. METHODS: Rural primary care practices (1,772) in 13 states (34.2% response) were surveyed in 2012 using mailed and Web-based questionnaires. FINDINGS: EHRs or HIT were used by 70% of respondents. Among practices using or intending to use the technology, most did not plan to hire new employees to obtain EHR/HIT skills and even fewer planned to hire consultants or vendors to fill gaps. Many practices had staff with some basic/entry, intermediate and/or advanced-level skills, but nearly two-thirds (61.4%) needed more staff training. Affordable access to vendors/consultants who understand their needs and availability of community college and baccalaureate-level training were the workforce-related barriers cited by the highest percentages of respondents. Accessing the Web/Internet challenged nearly a quarter of practices in isolated rural areas, and nearly a fifth in small rural areas. Finding relevant vendors/consultants and qualified staff were greater barriers in small and isolated rural areas than in large rural areas. DISCUSSION/CONCLUSIONS: Rural primary care practices mainly will rely on existing staff for continued implementation and use of EHR/HIT systems. Infrastructure and workforce-related barriers remain and must be overcome before practices can fully manage patient populations and exchange patient information among care system partners. Efforts to monitor adoption of these skills and ongoing support for continuing education will likely benefit rural populations.

Projected impact of the ICD-10-CM/PCS conversion on longitudinal data and the Joint Commission Core Measures.

The transition from ICD-9-CM to ICD-10-CM/PCS is expected to result in longitudinal data discontinuities, as occurred with cause-of-death in 1999. The General Equivalence Maps (GEMs), while useful for suggesting potential maps do not provide guidance regarding the frequency of any matches. Longitudinal data comparisons can only be reliable if they use comparability ratios or factors which have been calculated using records coded in both classification systems. This study utilized 3,969 de-identified dually coded records to examine raw comparability ratios, as well as the comparability ratios between the Joint Commission Core Measures. The raw comparability factor results range from 16.216 for Nicotine dependence, unspecified, uncomplicated to 118.009 for Chronic obstructive pulmonary disease, unspecified. The Joint Commission Core Measure comparability factor results range from 27.15 for Acute Respiratory Failure to 130.16 for Acute Myocardial Infarction. These results indicate significant differences in comparability between ICD-9-CM and ICD-10-CM code assignment, including when the codes are used for external reporting such as the Joint Commission Core Measures. To prevent errors in decision-making and reporting, all stakeholders relying on longitudinal data for measure reporting and other purposes should investigate the impact of the conversion on their data.

Preparing for ICD-10-CM/PCS implementation: impact on productivity and quality.

Coding productivity is expected to drop significantly during the lead-up to and in the initial stages of ICD-10-CM/PCS implementation, now expected to be delayed until October 1, 2015. This study examined the differences in coding productivity between ICD-9-CM and ICD-10-CM/PCS for hospital inpatient cases matched for complexity and severity. Additionally, interrater reliability was calculated to determine the quality of the coding. On average, coding of an inpatient record took 17.71 minutes (69 percent) longer with ICD-10-CM/PCS than with ICD-9-CM. A two-tailed T-test for statistical validity for independent samples was significant (p = .001). No coder characteristics such as years of experience or educational level were found to be a significant factor in coder productivity. Coders who had received more extensive training were faster than coders who had received only basic training. Though this difference was not statistically significant, it provides a strong indication of significant return on investment for staff training time. Coder interrater reliability was substantial for ICD-9-CM but only moderate for ICD-10-CM/PCS, though some ICD-10-CM/PCS cases had complete interrater (coder) agreement. Time spent coding a case was negatively correlated with interrater reliability (-0.425 for ICD-10-CM and -0.349 for ICD-10-PCS). This finding signals that increased time per case does not necessarily translate to higher quality. Adequate training for coders, as well as guidance regarding time invested per record, is important. Additionally, these findings indicate that previous estimates of initial coder productivity loss with ICD-10-CM/PCS may have been understated.