Development and validation of an instrument to evaluate the perspective of using the electronic health record in a hospital setting.

BACKGROUND: Evaluating healthcare information systems, such as the Electronic Health Records (EHR), is both challenging and essential, especially in resource-limited countries. This study aims to psychometrically develop and validate an instrument (questionnaire) to assess the factors influencing the successful adoption of the EHR system by healthcare professionals in Moroccan university hospitals. METHODS: The questionnaire validation process occurred in two main stages. Initially, data collected from a pilot sample of 164 participants underwent analysis using exploratory factor analysis (EFA) to evaluate the validity and reliability of the retained factor structure. Subsequently, the validity of the overall measurement model was confirmed using confirmatory factor analysis (CFA) in a sample of 368 healthcare professionals. RESULTS: The structure of the modified HOT-fit model, comprising seven constructs (System Quality, Information Quality, Information technology Service Quality, User Satisfaction, Organization, Environment, and Clinical Performance), was confirmed through confirmatory factor analysis. Absolute, incremental, and parsimonious fit indices all indicated an appropriate level of acceptability, affirming the robustness of the measurement model. Additionally, the instrument demonstrated adequate reliability and convergent validity, with composite reliability values ranging from 0.75 to 0.89 and average variance extracted (AVE) values ranging from 0.51 to 0.63. Furthermore, the square roots of AVE values exceeded the correlations between different pairs of constructs, and the heterotrait-monotrait ratio of correlations (HTMT) was below 0.85, confirming suitable discriminant validity. CONCLUSIONS: The resulting instrument, due to its rigorous development and validation process, can serve as a reliable and valid tool for assessing the success of information technologies in similar contexts.

Investigating the influencing factors of evidence-based healthcare practice adoption.

BACKGROUND:  In healthcare facilities, evidence-based healthcare practice (EBHP) is becoming more widely acknowledged as a critical element of patient care delivery. An increasingly important component of EBHP is the implementation of electronic health records (EHRs). OBJECTIVES:  This study aims to investigate factors that influence EBHP adoption in public healthcare institutions in South Africa. METHOD:  Four hundred and fifty patients were self-administered to healthcare professionals at an academic public hospital in Gauteng and used in this study. A total of 300 responses were available for use in the final analysis following the data cleaning procedure. Utilising structural equation modelling (SEM), the collected data were analysed. RESULTS:  Perceived ease of use (PEOU) and perceived usefulness (PU) were found to be major variables in the adoption of EBHP along with technological, organisational and environmental factors. The technology context relative advantage (RELA) was shown to have a positive significant influence on the adoption of evidence-based healthcare practice by the PEOU and PU, with the environmental context government laws and regulations (GLRS) and organisational context organisational readiness (ORGR) coming in second and third, respectively. CONCLUSION:  Perceived ease of use, PU, ORGR, and GLRS are regarded as a vital variables in the implementation of EBHP in South African public hospitals.Contribution: The study's conclusions would be helpful to policymakers as they redefine nursing practice. Furthermore, the findings heighten the consciousness of healthcare practitioners regarding the significance of employing evidence-based practice while making decisions.

Electronic Health Record Nudges and Health Care Quality and Outcomes in Primary Care: A Systematic Review.

Importance: Nudges have been increasingly studied as a tool for facilitating behavior change and may represent a novel way to modify the electronic health record (EHR) to encourage evidence-based care. Objective: To evaluate the association between EHR nudges and health care outcomes in primary care settings and describe implementation facilitators and barriers. Evidence Review: On June 9, 2023, an electronic search was performed in PubMed, Embase, PsycINFO, CINAHL, and Web of Science for all articles about clinician-facing EHR nudges. After reviewing titles, abstracts, and full texts, the present review was restricted to articles that used a randomized clinical trial (RCT) design, focused on primary care settings, and evaluated the association between EHR nudges and health care quality and patient outcome measures. Two reviewers abstracted the following elements: country, targeted clinician types, medical conditions studied, length of evaluation period, study design, sample size, intervention conditions, nudge mechanisms, implementation facilitators and barriers encountered, and major findings. The findings were qualitatively reported by type of health care quality and patient outcome and type of primary care condition targeted. The Risk of Bias 2.0 tool was adapted to evaluate the studies based on RCT design (cluster, parallel, crossover). Studies were scored from 0 to 5 points, with higher scores indicating lower risk of bias. Findings: Fifty-four studies met the inclusion criteria. Overall, most studies (79.6%) were assessed to have a moderate risk of bias. Most or all descriptive (eg, documentation patterns) (30 of 38) or patient-centeredness measures (4 of 4) had positive associations with EHR nudges. As for other measures of health care quality and patient outcomes, few had positive associations between EHR nudges and patient safety (4 of 12), effectiveness (19 of 48), efficiency (0 of 4), patient-reported outcomes (0 of 3), patient adherence (1 of 2), or clinical outcome measures (1 of 7). Conclusions and Relevance: This systematic review found low- and moderate-quality evidence that suggested that EHR nudges were associated with improved descriptive measures (eg, documentation patterns). Meanwhile, it was unclear whether EHR nudges were associated with improvements in other areas of health care quality, such as effectiveness and patient safety outcomes. Future research is needed using longer evaluation periods, a broader range of primary care conditions, and in deimplementation contexts.

Enhancing clinical data retrieval with Smart Watchers: a NiFi-based ETL pipeline for Elasticsearch queries.

BACKGROUND: The aim is to develop and deploy an automated clinical alert system to enhance patient care and streamline healthcare operations. Structured and unstructured data from multiple sources are used to generate near real-time alerts for specific clinical scenarios, with an additional goal to improve clinical decision-making through accuracy and reliability. METHODS: The automated clinical alert system, named Smart Watchers, was developed using Apache NiFi and Python scripts to create flexible data processing pipelines and customisable clinical alerts. A comparative analysis between Smart Watchers and the legacy Elastic Watchers was conducted to evaluate performance metrics such as accuracy, reliability, and scalability. The evaluation involved measuring the time taken for manual data extraction through the electronic patient record (EPR) front-end and comparing it with the automated data extraction process using Smart Watchers. RESULTS: Deployment of Smart Watchers showcased a consistent time savings between 90% to 98.67% compared to manual data extraction through the EPR front-end. The results demonstrate the efficiency of Smart Watchers in automating data extraction and alert generation, significantly reducing the time required for these tasks when compared to manual methods in a scalable manner. CONCLUSIONS: The research underscores the utility of employing an automated clinical alert system, and its portability facilitated its use across multiple clinical settings. The successful implementation and positive impact of the system lay a foundation for future technological innovations in this rapidly evolving field.

Real-world data to support post-market safety and performance of embolization coils: evidence generation from a medical device manufacturer and data institute partnership.

BACKGROUND: Recognizing the limitations of pre-market clinical data, regulatory authorities have embraced total product lifecycle management with post-market surveillance (PMS) data to assess medical device safety and performance. One method of proactive PMS involves the analysis of real-world data (RWD) through retrospective review of electronic health records (EHR). Because EHRs are patient-centered and focused on providing tools that clinicians use to determine care rather than collecting information on individual medical products, the process of transforming RWD into real-world evidence (RWE) can be laborious, particularly for medical devices with broad clinical use and extended clinical follow-up. This study describes a method to extract RWD from EHR to generate RWE on the safety and performance of embolization coils. METHODS: Through a partnership between a non-profit data institute and a medical device manufacturer, information on implantable embolization coils' use was extracted, linked, and analyzed from clinical data housed in an electronic data warehouse from the state of Indiana's largest health system. To evaluate the performance and safety of the embolization coils, technical success and safety were defined as per the Society of Interventional Radiology guidelines. A multi-prong strategy including electronic and manual review of unstructured (clinical chart notes) and structured data (International Classification of Disease codes), was developed to identify patients with relevant devices and extract data related to the endpoints. RESULTS: A total of 323 patients were identified as treated using Cook Medical Tornado, Nester, or MReye embolization coils between 1 January 2014 and 31 December 2018. Available clinical follow-up for these patients was 1127 ± 719 days. Indications for use, adverse events, and procedural success rates were identified via automated extraction of structured data along with review of available unstructured data. The overall technical success rate was 96.7%, and the safety events rate was 5.3% with 18 major adverse events in 17 patients. The calculated technical success and safety rates met pre-established performance goals (≥ 85% for technical success and ≤ 12% for safety), highlighting the relevance of this surveillance method. CONCLUSIONS: Generating RWE from RWD requires careful planning and execution. The process described herein provided valuable longitudinal data for PMS of real-world device safety and performance. This cost-effective approach can be translated to other medical devices and similar RWD database systems.

Differences in changes of data completeness after the implementation of an electronic medical record in three surgical departments of a German hospital-a longitudinal comparative document analysis.

PURPOSE: The European health data space promises an efficient environment for research and policy-making. However, this data space is dependent on high data quality. The implementation of electronic medical record systems has a positive impact on data quality, but improvements are not consistent across empirical studies. This study aims to analyze differences in the changes of data quality and to discuss these against distinct stages of the electronic medical record's adoption process. METHODS: Paper-based and electronic medical records from three surgical departments were compared, assessing changes in data quality after the implementation of an electronic medical record system. Data quality was operationalized as completeness of documentation. Ten information that must be documented in both record types (e.g. vital signs) were coded as 1 if they were documented, otherwise as 0. Chi-Square-Tests were used to compare percentage completeness of these ten information and t-tests to compare mean completeness per record type. RESULTS: A total of N = 659 records were analyzed. Overall, the average completeness improved in the electronic medical record, with a change from 6.02 (SD = 1.88) to 7.2 (SD = 1.77). At the information level, eight information improved, one deteriorated and one remained unchanged. At the level of departments, changes in data quality show expected differences. CONCLUSION: The study provides evidence that improvements in data quality could depend on the process how the electronic medical record is adopted in the affected department. Research is needed to further improve data quality through implementing new electronical medical record systems or updating existing ones.

Towards Standardizing Ophthalmic Data for Seamless Interoperability in Eye Care.

INTRODUCTION: Seamless interoperability of ophthalmic clinical data is beneficial for improving patient care and advancing research through the integration of data from various sources. Such consolidation increases the amount of data available, leading to more robust statistical analyses, and improving the accuracy and reliability of artificial intelligence models. However, the lack of consistent, harmonized data formats and meanings (syntactic and semantic interoperability) poses a significant challenge in sharing ophthalmic data. METHODS: The Health Level 7 (HL7) Fast Healthcare Interoperability Resources (FHIR), a standard for the exchange of healthcare data, emerges as a promising solution. To facilitate cross-site data exchange in research, the German Medical Informatics Initiative (MII) has developed a core data set (CDS) based on FHIR. RESULTS: This work investigates the suitability of the MII CDS specifications for exchanging ophthalmic clinical data necessary to train and validate a specific machine learning model designed for predicting visual acuity. In interdisciplinary collaborations, we identified and categorized the required ophthalmic clinical data and explored the possibility of its mapping to FHIR using the MII CDS specifications. DISCUSSION: We found that the current FHIR MII CDS specifications do not completely accommodate the ophthalmic clinical data we investigated, indicating that the creation of an extension module is essential.

lab2clean: a novel algorithm for automated cleaning of retrospective clinical laboratory results data for secondary uses.

BACKGROUND: The integrity of clinical research and machine learning models in healthcare heavily relies on the quality of underlying clinical laboratory data. However, the preprocessing of this data to ensure its reliability and accuracy remains a significant challenge due to variations in data recording and reporting standards. METHODS: We developed lab2clean, a novel algorithm aimed at automating and standardizing the cleaning of retrospective clinical laboratory results data. lab2clean was implemented as two R functions specifically designed to enhance data conformance and plausibility by standardizing result formats and validating result values. The functionality and performance of the algorithm were evaluated using two extensive electronic medical record (EMR) databases, encompassing various clinical settings. RESULTS: lab2clean effectively reduced the variability of laboratory results and identified potentially erroneous records. Upon deployment, it demonstrated effective and fast standardization and validation of substantial laboratory data records. The evaluation highlighted significant improvements in the conformance and plausibility of lab results, confirming the algorithm's efficacy in handling large-scale data sets. CONCLUSIONS: lab2clean addresses the challenge of preprocessing and cleaning clinical laboratory data, a critical step in ensuring high-quality data for research outcomes. It offers a straightforward, efficient tool for researchers, improving the quality of clinical laboratory data, a major portion of healthcare data. Thereby, enhancing the reliability and reproducibility of clinical research outcomes and clinical machine learning models. Future developments aim to broaden its functionality and accessibility, solidifying its vital role in healthcare data management.

Standardizing Corneal Transplantation Records Using openEHR: Case Study.

BACKGROUND: Corneal transplantation, also known as keratoplasty, is a widely performed surgical procedure that aims to restore vision in patients with corneal damage. The success of corneal transplantation relies on the accurate and timely management of patient information, which can be enhanced using electronic health records (EHRs). However, conventional EHRs are often fragmented and lack standardization, leading to difficulties in information access and sharing, increased medical errors, and decreased patient safety. In the wake of these problems, there is a growing demand for standardized EHRs that can ensure the accuracy and consistency of patient data across health care organizations. OBJECTIVE: This paper proposes the use of openEHR structures for standardizing corneal transplantation records. The main objective of this research was to improve the quality and interoperability of EHRs in corneal transplantation, making it easier for health care providers to capture, share, and analyze clinical information. METHODS: A series of sequential steps were carried out in this study to implement standardized clinical records using openEHR specifications. These specifications furnish a methodical approach that ascertains the development of high-quality clinical records. In broad terms, the methodology followed encompasses the conduction of meetings with health care professionals and the modeling of archetypes, templates, forms, decision rules, and work plans. RESULTS: This research resulted in a tailored solution that streamlines health care delivery and meets the needs of medical professionals involved in the corneal transplantation process while seamlessly aligning with contemporary clinical practices. The proposed solution culminated in the successful integration within a Portuguese hospital of 3 key components of openEHR specifications: forms, Decision Logic Modules, and Work Plans. A statistical analysis of data collected from May 1, 2022, to March 31, 2023, allowed for the perception of the use of the new technologies within the corneal transplantation workflow. Despite the completion rate being only 63.9% (530/830), which can be explained by external factors such as patient health and availability of donor organs, there was an overall improvement in terms of task control and follow-up of the patients' clinical process. CONCLUSIONS: This study shows that the adoption of openEHR structures represents a significant step forward in the standardization and optimization of corneal transplantation records. It offers a detailed demonstration of how to implement openEHR specifications and highlights the different advantages of standardizing EHRs in the field of corneal transplantation. Furthermore, it serves as a valuable reference for researchers and practitioners who are interested in advancing and improving the exploitation of EHRs in health care.

AI-Powered Clinical Documentation and Clinicians' Electronic Health Record Experience: A Nonrandomized Clinical Trial.

This nonrandomized clinical trial investigated the electronic health record (EHR) experiences of clinicians before and after implementation of an artificial intelligence (AI)­powered clinical documentation tool.

Electronic nursing documentation for patient safety, quality of nursing care, and documentation: a systematic review .

Objective: To evaluate the impact of electronic nursing documentation on patient safety, quality of nursing care and documentation. METHODS: The systematic review was conducted in December 2022, and comprised a comprehensive search on Scopus, ScienceDirect, ProQuest, PubMed, Cumulative Index to Nursing and Allied Health Literature, Sage Journals and Google Scholar databases for English-language human studies published between 2018 and 2022. The key words used in the search included "Nursing", "care", "documentation", "record", "electronic", "process" and "health services". The risk of bias was assessed using Strengthening the Reporting of Observational Studies in Epidemiology tool. RESULTS: Of the 469 items initially identified, 15(3.2%) were analysed in detail, indicating a positive influence of electronic nursing documentation on patient safety, care quality, and documentation. However, shortcomings were observed in the development of electronic nursing documentation for optimal effectiveness. Conclusion: Electronic nursing documentation significantly enhanced patient safety, care quality and documentation. To facilitate its integration into clinical settings, a standardised and logically structured electronic nursing documentation system is essential.

Reducing Failed Extubations in Preterm Infants Via Standardization and Real-Time Decision Support.

BACKGROUND AND OBJECTIVES: Failed extubations are associated with pulmonary morbidity in hospitalized premature newborns. The objective of this study was to use quality improvement methodology to reduce failed extubations through practice standardization and integrating a real-time extubation success calculator into the electronic medical record (EMR). METHODS: A specific, measurable, achievable, relevant, and time-bound aim was developed to reduce failed extubations (defined as reintubation <5 days from primary extubation) by 50% among infants <32 weeks' gestational age (GA) or <1500 g birth weight by December 31, 2022. Plan-do-study-act cycles were developed to standardize postextubation respiratory support and integrate the EMR-based calculator. Outcome measures included extubation failure rates. Balancing measures included days on mechanical ventilation and number of patients intubated <3 days. Process measures were followed for guideline compliance. Statistical process control charts were used to track time-ordered data and detect special cause variation. RESULTS: We observed a reduction in failed extubations from 10.3% to 2.3%, with special cause variation noted after both plan-do-study-act cycle #1 and #2. Special cause variation was detected in both GA subgroups: <28 weeks' GA (22.0%-8.6%) and ≥28 weeks' GA (4.6%-0.3%). Additionally, the average number of infants intubated <3 days increased (60.2%-73.6%), whereas average ventilator days decreased (10.8-7.0). Finally, the time from infants' extubation score reaching threshold (≥60%) to extubation decreased (14.1-6.4 days) after launching the EMR-integrated calculator. CONCLUSIONS: Practice standardization and implementation of an EMR-based real-time clinical decision support tool improved extubation success, promoted earlier extubation, and reduced ventilator days in premature newborns.

Special supplement issue on quality assurance and enrichment of biological and biomedical ontologies and terminologies.

Ontologies and terminologies serve as the backbone of knowledge representation in biomedical domains, facilitating data integration, interoperability, and semantic understanding across diverse applications. However, the quality assurance and enrichment of these resources remain an ongoing challenge due to the dynamic nature of biomedical knowledge. In this editorial, we provide an introductory summary of seven articles included in this special supplement issue for quality assurance and enrichment of biological and biomedical ontologies and terminologies. These articles span a spectrum of topics, such as development of automated quality assessment frameworks for Resource Description Framework (RDF) resources, identification of missing concepts in SNOMED CT through logical definitions, and developing a COVID interface terminology to enable automatic annotations of COVID-19 related Electronic Health Records (EHRs). Collectively, these contributions underscore the ongoing efforts to improve the accuracy, consistency, and interoperability of biomedical ontologies and terminologies, thus advancing their pivotal role in healthcare and biomedical research.

Improving the quality of Persian clinical text with a novel spelling correction system.

BACKGROUND: The accuracy of spelling in Electronic Health Records (EHRs) is a critical factor for efficient clinical care, research, and ensuring patient safety. The Persian language, with its abundant vocabulary and complex characteristics, poses unique challenges for real-word error correction. This research aimed to develop an innovative approach for detecting and correcting spelling errors in Persian clinical text. METHODS: Our strategy employs a state-of-the-art pre-trained model that has been meticulously fine-tuned specifically for the task of spelling correction in the Persian clinical domain. This model is complemented by an innovative orthographic similarity matching algorithm, PERTO, which uses visual similarity of characters for ranking correction candidates. RESULTS: The evaluation of our approach demonstrated its robustness and precision in detecting and rectifying word errors in Persian clinical text. In terms of non-word error correction, our model achieved an F1-Score of 90.0% when the PERTO algorithm was employed. For real-word error detection, our model demonstrated its highest performance, achieving an F1-Score of 90.6%. Furthermore, the model reached its highest F1-Score of 91.5% for real-word error correction when the PERTO algorithm was employed. CONCLUSIONS: Despite certain limitations, our method represents a substantial advancement in the field of spelling error detection and correction for Persian clinical text. By effectively addressing the unique challenges posed by the Persian language, our approach paves the way for more accurate and efficient clinical documentation, contributing to improved patient care and safety. Future research could explore its use in other areas of the Persian medical domain, enhancing its impact and utility.

Enhancing the Management of Deployment-Limiting Medical Conditions in the U.S. Department of Navy Leveraging Electronic Health Records and Predictive Analytics Methods.

INTRODUCTION: Deployment-limiting medical conditions (DLMCs) such as debilitating injuries and conditions may interfere with the ability of military service members (SMs) to deploy. SMs in the United States (U.S.) Department of the Navy (DoN) with DLMCs who are not deployable should be placed in the medically restricted status of limited duty (LIMDU) or referred to the Physical Evaluation Board (PEB) for Service retention determination. It is critical to identify SMs correctly and promptly with DLMCs and predict their return-to-duty (RTD) to ensure the combat readiness of the U.S. Military. In this study, an algorithmic approach was developed to identify DoN SMs with previously unidentified DLMCs and predict whether SMs on LIMDU will be able to RTD. MATERIALS AND METHODS: Five years of historical data (2016-2022) were obtained from inpatient and outpatient datasets across direct and purchased care from the Military Health System (MHS) Data Repository (MDR). Key fields included International Classification of Diseases diagnosis and procedure codes, Current Procedure Terminology codes, prescription medications, and demographics information such as age, rank, gender, and service. The data consisted of 44,580,668 medical encounters across 1,065,224 SMs. To identify SMs with unidentified DLMCs, we developed an ensemble model combining outputs from multiple machine learning (ML) algorithms. When the ML ensemble model predicted a SM to have high risk scores, despite appearing healthy on administrative reports, their case was reviewed by expert clinicians to investigate for previously unidentified DLMCs; and such feedback served to validate the developed algorithms. In addition, leveraging 1,735,422 encounters (60,433 SMs) from LIMDU periods, we developed four separate ML models to estimate RTD probabilities for SMs after each medical encounter and predict the final LIMDU outcome. RESULTS: The ensemble model had 0.91 area under the receiver operating characteristic curve (AUROC). Out of 236 (round one) and 314 (round two) SMs reviewed by clinicians, 127 (54%) and 208 (66%) SMs were identified with a previously unidentified or undocumented DLMC, respectively. Regarding predicting RTD for SMs placed on LIMDU, the best performing ML model achieved 0.76 AUROC, 68% sensitivity, and 71% specificity. CONCLUSION: Our research highlighted potential benefits of using predictive analytics in a medical assessment to identify SMs with DLMCs and to predict RTD outcomes once placed on LIMDU. This capability is being deployed for real-time clinical decision support to enhance health care provider's deployability assessment capability, improve accuracy of the DLMC population, and enhance combat readiness of the U.S Military.

Supplementing systematic review findings with healthcare system data: pilot projects from the Agency for Healthcare Research and Quality Evidence-based Practice Center program.

OBJECTIVES: The US Agency for Healthcare Research and Quality, through the Evidence-based Practice Center (EPC) Program, aims to provide health system decision makers with the highest-quality evidence to inform clinical decisions. However, limitations in the literature may lead to inconclusive findings in EPC systematic reviews (SRs). The EPC Program conducted pilot projects to understand the feasibility, benefits, and challenges of utilizing health system data to augment SR findings to support confidence in healthcare decision-making based on real-world experiences. STUDY DESIGN AND SETTING: Three contractors (each an EPC located at a different health system) selected a recently completed SR conducted by their center and identified an evidence gap that electronic health record (EHR) data might address. All pilot project topics addressed clinical questions as opposed to care delivery, care organization, or care disparities topics that are common in EPC reports. Topic areas addressed by each EPC included infantile epilepsy, migraine, and hip fracture. EPCs also tracked additional resources needed to conduct supplemental analyses. The workgroup met monthly in 2022-2023 to discuss challenges and lessons learned from the pilot projects. RESULTS: Two supplemental data analyses filled an evidence gap identified in the SRs (raised certainty of evidence, improved applicability) and the third filled a health system knowledge gap. Project challenges fell under three themes: regulatory and logistical issues, data collection and analysis, and interpretation and presentation of findings. Limited ability to capture key clinical variables given inconsistent or missing data within the EHR was a major limitation. The workgroup found that conducting supplemental data analysis alongside an SR was feasible but adds considerable time and resources to the review process (estimated total hours to complete pilot projects ranged from 283 to 595 across EPCs), and that the increased effort and resources added limited incremental value. CONCLUSION: Supplementing existing SRs with analyses of EHR data is resource intensive and requires specialized skillsets throughout the process. While using EHR data for research has immense potential to generate real-world evidence and fill knowledge gaps, these data may not yet be ready for routine use alongside SRs.

Benchmarking Emergency Physician EHR Time per Encounter Based on Patient and Clinical Factors.

This cross-sectional study assesses the associations between patient and clinical factors and variations in time emergency department physicians spend using electronic health record (EHR) systems.

A human centered design approach to define and measure documentation quality using an EHR virtual simulation.

Electronic health record (EHR) documentation serves multiple functions, including recording patient health status, enabling interprofessional communication, supporting billing, and providing data to support the quality infrastructure of a Learning Healthcare System. There is no definition and standardized method to assess documentation quality in EHRs. Using a human-centered design (HCD) approach, we define and describe a method to measure documentation quality. Documentation quality was defined as timely, accurate, user-centered, and efficient. Measurement of quality used a virtual simulated standardized patient visit via an EHR vendor platform. By observing and recording documentation efforts, nurse practitioners (NPs) (N = 12) documented the delivery of an Age-Friendly Health System (AFHS) 4Ms (what Matters, Medication, Mentation, and Mobility) clinic visit using a standardized case. Results for timely documentation indicated considerable variability in completion times of documenting the 4Ms. Accuracy varied, as there were many types of episodes of erroneous documentation and extra time in seconds in documenting the 4Ms. The type and frequency of erroneous documentation efforts were related to navigation burden when navigating to different documentation tabs. The evaluated system demonstrated poor usability, with most participants scoring between 60 and 70 on the System Usability Scale (SUS). Efficiency, measured as click burden (the number of clicks used to navigate through a software system), revealed significant variability in the number of clicks required, with the NPs averaging approximately 13 clicks above the minimum requirement. The HCD methodology used in this study to assess the documentation quality proved feasible and provided valuable information on the quality of documentation. By assessing the quality of documentation, the gathered data can be leveraged to enhance documentation, optimize user experience, and elevate the quality of data within a Learning Healthcare System.