Contextual property detection in Dutch diagnosis descriptions for uncertainty, laterality and temporality.

BACKGROUND: Accurate, coded problem lists are valuable for data reuse, including clinical decision support and research. However, healthcare providers frequently modify coded diagnoses by including or removing common contextual properties in free-text diagnosis descriptions: uncertainty (suspected glaucoma), laterality (left glaucoma) and temporality (glaucoma 2002). These contextual properties could cause a difference in meaning between underlying diagnosis codes and modified descriptions, inhibiting data reuse. We therefore aimed to develop and evaluate an algorithm to identify these contextual properties. METHODS: A rule-based algorithm called UnLaTem (Uncertainty, Laterality, Temporality) was developed using a single-center dataset, including 288,935 diagnosis descriptions, of which 73,280 (25.4%) were modified by healthcare providers. Internal validation of the algorithm was conducted with an independent sample of 980 unique records. A second validation of the algorithm was conducted with 996 records from a Dutch multicenter dataset including 175,210 modified descriptions of five hospitals. Two researchers independently annotated the two validation samples. Performance of the algorithm was determined using means of the recall and precision of the validation samples. The algorithm was applied to the multicenter dataset to determine the actual prevalence of the contextual properties within the modified descriptions per specialty. RESULTS: For the single-center dataset recall (and precision) for removal of uncertainty, uncertainty, laterality and temporality respectively were 100 (60.0), 99.1 (89.9), 100 (97.3) and 97.6 (97.6). For the multicenter dataset for removal of uncertainty, uncertainty, laterality and temporality it was 57.1 (88.9), 86.3 (88.9), 99.7 (93.5) and 96.8 (90.1). Within the modified descriptions of the multicenter dataset, 1.3% contained removal of uncertainty, 9.9% uncertainty, 31.4% laterality and 9.8% temporality. CONCLUSIONS: We successfully developed a rule-based algorithm named UnLaTem to identify contextual properties in Dutch modified diagnosis descriptions. UnLaTem could be extended with more trigger terms, new rules and the recognition of term order to increase the performance even further. The algorithm's rules are available as additional file 2. Implementing UnLaTem in Dutch hospital systems can improve precision of information retrieval and extraction from diagnosis descriptions, which can be used for data reuse purposes such as decision support and research.

Correctly structured problem lists lead to better and faster clinical decision-making in electronic health records compared to non-curated problem lists: A single-blinded crossover randomized controlled trial.

BACKGROUND: Correctly structured problem lists in electronic health records (EHRs) offer major benefits to patient care. Without structured lists, diagnosis information is often scatteredly documented in free text, which may contribute to errors and inefficient information retrieval. This study aims to assess whether EHRs with correctly structured problem lists result in better and faster clinical decision-making compared to non-curated problem lists. METHODS: Two versions of two patient records (A and B) were created in an EHR training environment: one version included diagnosis information structured and coded on the problem list ("correctly structured problem list"), the other version had missing problem list diagnoses and diagnosis information partly documented in free text ("non-curated problem list"). In this single-blinded crossover randomized controlled trial, healthcare providers, who can prescribe medications, from two Dutch university medical center locations first evaluated a randomized version of patient A, then B. Participants were asked to motivate their answer to two medication prescription questions. One (test) question required information similarly presented in both record versions. The second (comparison) question required information documented on problem lists and/or in notes. The primary outcome measure was the correctness of the motivated answer to the comparison question. Secondary outcome measure was the time to answer and motivate both questions correctly. RESULTS: As planned, 160 participants enrolled. Two were excluded for not meeting inclusion criteria. Correctly structured problem lists increased providers' ability to answer the comparison question correctly (56.3 % versus 33.5 %, McNemar odds ratio 2.80 (1.65-4.93) 95 %-CI). Median time to answer both questions correctly was significantly lower for EHRs with correctly structured problem lists (Wilcoxon-signed-rank test p = 0.00002, with incorrect answers coded equally at slowest time). CONCLUSIONS: Correctly structured problem lists lead to better and faster clinical decision-making. Increased structured problem lists usage may be warranted for which implementation policies should be developed.

Inaccurate recording of routinely collected data items influences identification of COVID-19 patients.

BACKGROUND: During the Coronavirus disease 2019 (COVID-19) pandemic it became apparent that it is difficult to extract standardized Electronic Health Record (EHR) data for secondary purposes like public health decision-making. Accurate recording of, for example, standardized diagnosis codes and test results is required to identify all COVID-19 patients. This study aimed to investigate if specific combinations of routinely collected data items for COVID-19 can be used to identify an accurate set of intensive care unit (ICU)-admitted COVID-19 patients. METHODS: The following routinely collected EHR data items to identify COVID-19 patients were evaluated: positive reverse transcription polymerase chain reaction (RT-PCR) test results; problem list codes for COVID-19 registered by healthcare professionals and COVID-19 infection labels. COVID-19 codes registered by clinical coders retrospectively after discharge were also evaluated. A gold standard dataset was created by evaluating two datasets of suspected and confirmed COVID-19-patients admitted to the ICU at a Dutch university hospital between February 2020 and December 2020, of which one set was manually maintained by intensivists and one set was extracted from the EHR by a research data management department. Patients were labeled 'COVID-19' if their EHR record showed diagnosing COVID-19 during or right before an ICU-admission. Patients were labeled 'non-COVID-19' if the record indicated no COVID-19, exclusion or only suspicion during or right before an ICU-admission or if COVID-19 was diagnosed and cured during non-ICU episodes of the hospitalization in which an ICU-admission took place. Performance was determined for 37 queries including real-time and retrospective data items. We used the F1 score, which is the harmonic mean between precision and recall. The gold standard dataset was split into one subset including admissions between February and April and one subset including admissions between May and December to determine accuracy differences. RESULTS: The total dataset consisted of 402 patients: 196 'COVID-19' and 206 'non-COVID-19' patients. F1 scores of search queries including EHR data items that can be extracted real-time ranged between 0.68 and 0.97 and for search queries including the data item that was retrospectively registered by clinical coders F1 scores ranged between 0.73 and 0.99. F1 scores showed no clear pattern in variability between the two time periods. CONCLUSIONS: Our study showed that one cannot rely on individual routinely collected data items such as coded COVID-19 on problem lists to identify all COVID-19 patients. If information is not required real-time, medical coding from clinical coders is most reliable. Researchers should be transparent about their methods used to extract data. To maximize the ability to completely identify all COVID-19 cases alerts for inconsistent data and policies for standardized data capture could enable reliable data reuse.

Development of a Framework for Redesigning a Terminology Maintenance Process - Case Study in the Netherlands.

Structuring clinical data in electronic health records supports reuse of data to improve quality of care, reduce costs and perform research. This requires terminologies to assign terms from language used in a specific domain to medical concepts. Given the developing character of medical knowledge, these terminologies need continuous maintenance. Nonetheless, little is known about terminology maintenance processes. To specify the (re)design of a terminology maintenance process, we first merged and adapted two static theoretical frameworks that consisted of criteria relating to using a terminology, divided among relevant stakeholders. Following, we applied the framework to the healthcare terminology maintenance process in the Netherlands. We held interviews with relevant stakeholders and used the framework as checklist to identify missing criteria and bottlenecks. Saturation in interviews and fulfilment of the criteria indicated that all bottlenecks were discovered, therefore the framework was considered useful for redesigning a terminology maintenance process. Other countries could benefit from this framework as well to discover and resolve any unfulfilled maintenance criteria.

Factors Influencing Problem List Use in Electronic Health Records-Application of the Unified Theory of Acceptance and Use of Technology.

BACKGROUND: Problem-oriented electronic health record (EHR) systems can help physicians to track a patient's status and progress, and organize clinical documentation, which could help improving quality of clinical data and enable data reuse. The problem list is central in a problem-oriented medical record. However, current problem lists remain incomplete because of the lack of end-user training and inaccurate content of underlying terminologies. This leads to modifications of diagnosis code descriptions and use of free-text notes, limiting reuse of data. OBJECTIVES: We aimed to investigate factors that influence acceptance and actual use of the problem list, and used these to propose recommendations, to increase the value of problem lists for (re)use. METHODS: Semistructured interviews were conducted with physicians, heads of medical departments, and data quality experts, who were invited through snowball sampling. The interviews were transcribed and coded. Comments were fitted in constructs of the validated framework unified theory of acceptance user technology (UTAUT), and were discussed in terms of facilitators and barriers. RESULTS: In total, 24 interviews were conducted. We found large variability in attitudes toward problem list use. Barriers included uncertainty about the responsibility for maintaining the problem list and little perceived benefits. Facilitators included the (re)design of policies, improved (peer-to-peer) training to increase motivation, and positive peer feedback and monitoring. Motivation is best increased through sharing benefits relevant in the care process, such as providing overview, timely generation of discharge or referral letters, and reuse of data. Furthermore, content of the underlying terminology should be improved and the problem list should be better presented in the EHR system. CONCLUSION: To let physicians accept and use the problem list, policies and guidelines should be redesigned, and prioritized by supervising staff. Additionally, peer-to-peer training on the benefits of using the problem list is needed.