Role of Nursing Informatics in the Automation of Pneumonia Quality Measure Data Elements.

Core measures are standard metrics to reflect the processes of care provided by hospitals. Hospitals in the United States are expected to extract data from electronic health records, automated computation of core measures, and electronic submission of the quality measures data. Traditional manual calculation processes are time intensive and susceptible to error. Automated calculation has the potential to provide timely, accurate information, which could guide quality-of-care decisions, but this vision has yet to be achieved. In this study, nursing informaticists and data analysts implemented a method to automatically extract data elements from electronic health records to calculate a core measure. We analyzed the sensitivity, specificity, and accuracy of core measure data elements extracted via SQL query and compared the results to manually extracted data elements. This method achieved excellent performance for the structured data elements but was less efficient for semistructured and unstructured elements. We analyzed challenges in automating the calculation of quality measures and proposed a rule-based (hybrid) approach for semistructured and unstructured data elements.

An Evolving Ecosystem for Natural Language Processing in Department of Veterans Affairs.

In an ideal clinical Natural Language Processing (NLP) ecosystem, researchers and developers would be able to collaborate with others, undertake validation of NLP systems, components, and related resources, and disseminate them. We captured requirements and formative evaluation data from the Veterans Affairs (VA) Clinical NLP Ecosystem stakeholders using semi-structured interviews and meeting discussions. We developed a coding rubric to code interviews. We assessed inter-coder reliability using percent agreement and the kappa statistic. We undertook 15 interviews and held two workshop discussions. The main areas of requirements related to; design and functionality, resources, and information. Stakeholders also confirmed the vision of the second generation of the Ecosystem and recommendations included; adding mechanisms to better understand terms, measuring collaboration to demonstrate value, and datasets/tools to navigate spelling errors with consumer language, among others. Stakeholders also recommended capability to: communicate with developers working on the next version of the VA electronic health record (VistA Evolution), provide a mechanism to automatically monitor download of tools and to automatically provide a summary of the downloads to Ecosystem contributors and funders. After three rounds of coding and discussion, we determined the percent agreement of two coders to be 97.2% and the kappa to be 0.7851. The vision of the VA Clinical NLP Ecosystem met stakeholder needs. Interviews and discussion provided key requirements that inform the design of the VA Clinical NLP Ecosystem.

A human centered design approach to define and measure documentation quality using an EHR virtual simulation.

Electronic health record (EHR) documentation serves multiple functions, including recording patient health status, enabling interprofessional communication, supporting billing, and providing data to support the quality infrastructure of a Learning Healthcare System. There is no definition and standardized method to assess documentation quality in EHRs. Using a human-centered design (HCD) approach, we define and describe a method to measure documentation quality. Documentation quality was defined as timely, accurate, user-centered, and efficient. Measurement of quality used a virtual simulated standardized patient visit via an EHR vendor platform. By observing and recording documentation efforts, nurse practitioners (NPs) (N = 12) documented the delivery of an Age-Friendly Health System (AFHS) 4Ms (what Matters, Medication, Mentation, and Mobility) clinic visit using a standardized case. Results for timely documentation indicated considerable variability in completion times of documenting the 4Ms. Accuracy varied, as there were many types of episodes of erroneous documentation and extra time in seconds in documenting the 4Ms. The type and frequency of erroneous documentation efforts were related to navigation burden when navigating to different documentation tabs. The evaluated system demonstrated poor usability, with most participants scoring between 60 and 70 on the System Usability Scale (SUS). Efficiency, measured as click burden (the number of clicks used to navigate through a software system), revealed significant variability in the number of clicks required, with the NPs averaging approximately 13 clicks above the minimum requirement. The HCD methodology used in this study to assess the documentation quality proved feasible and provided valuable information on the quality of documentation. By assessing the quality of documentation, the gathered data can be leveraged to enhance documentation, optimize user experience, and elevate the quality of data within a Learning Healthcare System.

Shifting Culture Toward Age-Friendly Care: Lessons From VHA Early Adopters.

Background: The need for a health care workforce with expanded skills in the care of older adults is increasingly evident as the US population ages. The evidence-based Age-Friendly Health Systems (AFHS) framework establishes a structure to reliably assess and deliver effective care of older adults with multiple chronic conditions: what matters, medication, mentation, and mobility (4Ms). Half of veterans receiving Veterans Health Administration (VHA) care are aged ≥ 65 years, driving its transformation into the largest AFHS in the US. In this article, we offer lessons on the challenges to AFHS delivery and suggest opportunities to sustaining age-friendly care. Observations: Within 3 months of implementation, 85% to 100% of patients received 4M care in all care settings at our VA facilities. Key lessons learned include the importance of identifying, preparing, and supporting a champion to lead this effort; garnering facility and system leadership support at the outset; and integration with the electronic health record (EHR) for reliable and efficient data capture, reporting, and feedback. Although the goal is to establish AFHS in all care settings, we believe that initially including a geriatrics-focused care setting helped early adoption of 4Ms care in the sites described here. Conclusions: Early adopters at 2 VHA health care systems demonstrated successful AFHS implementation spanning different VHA facilities and care settings. Successful growth and sustainability may require leveraging the EHR to reduce documentation burden, increase standardization in care, and automate feedback, tracking, and reporting. A coordinated effort is underway to integrate AFHS into VHA documentation, performance evaluation, and metrics in both the legacy and new Cerner EHRs.

Automating Quality Measures for Heart Failure Using Natural Language Processing: A Descriptive Study in the Department of Veterans Affairs.

BACKGROUND: We developed an accurate, stakeholder-informed, automated, natural language processing (NLP) system to measure the quality of heart failure (HF) inpatient care, and explored the potential for adoption of this system within an integrated health care system. OBJECTIVE: To accurately automate a United States Department of Veterans Affairs (VA) quality measure for inpatients with HF. METHODS: We automated the HF quality measure Congestive Heart Failure Inpatient Measure 19 (CHI19) that identifies whether a given patient has left ventricular ejection fraction (LVEF) <40%, and if so, whether an angiotensin-converting enzyme inhibitor or angiotensin-receptor blocker was prescribed at discharge if there were no contraindications. We used documents from 1083 unique inpatients from eight VA medical centers to develop a reference standard (RS) to train (n=314) and test (n=769) the Congestive Heart Failure Information Extraction Framework (CHIEF). We also conducted semi-structured interviews (n=15) for stakeholder feedback on implementation of the CHIEF. RESULTS: The CHIEF classified each hospitalization in the test set with a sensitivity (SN) of 98.9% and positive predictive value of 98.7%, compared with an RS and SN of 98.5% for available External Peer Review Program assessments. Of the 1083 patients available for the NLP system, the CHIEF evaluated and classified 100% of cases. Stakeholders identified potential implementation facilitators and clinical uses of the CHIEF. CONCLUSIONS: The CHIEF provided complete data for all patients in the cohort and could potentially improve the efficiency, timeliness, and utility of HF quality measurements.