Don Lindberg's influence on future generations: The U.S. National Library of Medicine's biomedical informatics research training programs.

Through his visionary leadership as Director of the U.S. National Library of Medicine (NLM), Donald A. B. Lindberg M.D. influenced future generations of informatics professionals and the field of biomedical informatics itself. This chapter describes Dr. Lindberg's role in sponsoring and shaping the NLM's Institutional T15 training programs.

Donald A.B. Lindberg and the U.S. National Library of Medicine transformed biomedical and health informatics.

This overview summary of the Informatics Section of the book Transforming biomedical informatics and health information access: Don Lindberg and the U.S. National Library of Medicine illustrates how the NLM revolutionized the field of biomedical and health informatics during Lindberg's term as NLM Director. Authors present a before-and-after perspective of what changed, how it changed, and the impact of those changes.

A method for analyzing inpatient care variability through physicians' orders.

OBJECTIVE: Administrators assess care variability through chart review or cost variability to inform care standardization efforts. Chart review is costly and cost variability is imprecise. This study explores the potential of physician orders as an alternative measure of care variability. MATERIALS & METHODS: The authors constructed an order variability metric from adult Vanderbilt University Hospital patients treated between 2013 and 2016. The study compared how well a cost variability model predicts variability in the length of stay compared to an order variability model. Both models adjusted for covariates such as severity of illness, comorbidities, and hospital transfers. RESULTS: The order variability model significantly minimized the Akaike information criterion (superior outcome) compared to the cost variability model. This result also held when excluding patients who received intensive care. CONCLUSION: Order variability can potentially typify care variability better than cost variability. Order variability is a scalable metric, calculable during the course of care.

Discovering hidden knowledge through auditing clinical diagnostic knowledge bases.

OBJECTIVE: Evaluate potential for data mining auditing techniques to identify hidden concepts in diagnostic knowledge bases (KB). Improving completeness enhances KB applications such as differential diagnosis and patient case simulation. MATERIALS AND METHODS: Authors used unsupervised (Pearson's correlation - PC, Kendall's correlation - KC, and a heuristic algorithm - HA) methods to identify existing and discover new finding-finding interrelationships ("properties") in the INTERNIST-1/QMR KB. Authors estimated KB maintenance efficiency gains (effort reduction) of the approaches. RESULTS: The methods discovered new properties at 95% CI rates of [0.1%, 5.4%] (PC), [2.8%, 12.5%] (KC), and [5.6%, 18.8%] (HA). Estimated manual effort reduction for HA-assisted determination of new properties was approximately 50-fold. CONCLUSION: Data mining can provide an efficient supplement to ensuring the completeness of finding-finding interdependencies in diagnostic knowledge bases. Authors' findings should be applicable to other diagnostic systems that record finding frequencies within diseases (e.g., DXplain, ISABEL).

Two complementary AI approaches for predicting UMLS semantic group assignment: heuristic reasoning and deep learning.

OBJECTIVE: Use heuristic, deep learning (DL), and hybrid AI methods to predict semantic group (SG) assignments for new UMLS Metathesaurus atoms, with target accuracy ≥95%. MATERIALS AND METHODS: We used train-test datasets from successive 2020AA-2022AB UMLS Metathesaurus releases. Our heuristic "waterfall" approach employed a sequence of 7 different SG prediction methods. Atoms not qualifying for a method were passed on to the next method. The DL approach generated BioWordVec and SapBERT embeddings for atom names, BioWordVec embeddings for source vocabulary names, and BioWordVec embeddings for atom names of the second-to-top nodes of an atom's source hierarchy. We fed a concatenation of the 4 embeddings into a fully connected multilayer neural network with an output layer of 15 nodes (one for each SG). For both approaches, we developed methods to estimate the probability that their predicted SG for an atom would be correct. Based on these estimations, we developed 2 hybrid SG prediction methods combining the strengths of heuristic and DL methods. RESULTS: The heuristic waterfall approach accurately predicted 94.3% of SGs for 1 563 692 new unseen atoms. The DL accuracy on the same dataset was also 94.3%. The hybrid approaches achieved an average accuracy of 96.5%. CONCLUSION: Our study demonstrated that AI methods can predict SG assignments for new UMLS atoms with sufficient accuracy to be potentially useful as an intermediate step in the time-consuming task of assigning new atoms to UMLS concepts. We showed that for SG prediction, combining heuristic methods and DL methods can produce better results than either alone.

Donald A.B. Lindberg and the U.S. National Library of Medicine Transformed Biomedical and Health Informatics.

This overview summary of the Informatics Section of the book Transforming biomedical informatics and health information access: Don Lindberg and the U.S. National Library of Medicine illustrates how the NLM revolutionized the field of biomedical and health informatics during Lindberg's term as NLM Director. Authors present a before-and-after perspective of what changed, how it changed, and the impact of those changes.

Don Lindberg's Influence on Future Generations: The U.S. National Library of Medicine's Biomedical Informatics Research Training Programs.

Through his visionary leadership as Director of the U.S. National Library of Medicine (NLM), Donald A.B. Lindberg M.D. influenced future generations of informatics professionals and the field of biomedical informatics itself. This chapter describes Dr. Lindberg's role in sponsoring and shaping the NLM's Institutional T15 training programs.

Do electronic health record systems "dumb down" clinicians?

A panel sponsored by the American College of Medical Informatics (ACMI) at the 2021 AMIA Symposium addressed the provocative question: "Are Electronic Health Records dumbing down clinicians?" After reviewing electronic health record (EHR) development and evolution, the panel discussed how EHR use can impair care delivery. Both suboptimal functionality during EHR use and longer-term effects outside of EHR use can reduce clinicians' efficiencies, reasoning abilities, and knowledge. Panel members explored potential solutions to problems discussed. Progress will require significant engagement from clinician-users, educators, health systems, commercial vendors, regulators, and policy makers. Future EHR systems must become more user-focused and scalable and enable providers to work smarter to deliver improved care.

Analyses of longitudinal, hospital clinical laboratory data with application to blood glucose concentrations.

Electronic medical record (EMR) systems afford researchers with opportunities to investigate a broad range of scientific questions. In contrast to purposeful study designs, however, EMR data acquisition procedures typically do not align with any specific hypothesis. Subsequent investigations therefore require detailed characterization of clinical procedures and protocols that underlie EMR data, as well as careful consideration of model choice. For example, many intensive care units currently implement insulin infusion protocols to better control patients' blood glucose levels. The protocols use prior glucose levels to determine, in part, how to adjust the infusion rate. Such feedback loops introduce time-dependent confounding into longitudinal analyses even though they may not always be evident to the analyst. In this paper, we review commonly used longitudinal model specifications and interpretations and show how these are particularly important in the presence of hospital-based clinical protocols. We show that parameter relationships among various models can be used to identify and characterize the impact of time-dependent confounding and therefore help explain seemingly incongruous conclusions. We also review important estimation challenges in the presence of time-dependent confounding and show how certain model specifications may be more or less susceptible to bias. To illustrate these points, we present a detailed analysis of the relationship between blood glucose levels and insulin doses on the basis of data from an intensive care unit.

The roles of the US National Library of Medicine and Donald A.B. Lindberg in revolutionizing biomedical and health informatics.

Over a 31-year span as Director of the US National Library of Medicine (NLM), Donald A.B. Lindberg, MD, and his extraordinary NLM colleagues fundamentally changed the field of biomedical and health informatics-with a resulting impact on biomedicine that is much broader than its influence on any single subfield. This article provides substance to bolster that claim. The review is based in part on the informatics section of a new book, "Transforming biomedical informatics and health information access: Don Lindberg and the US National Library of Medicine" (IOS Press, forthcoming 2021). After providing insights into selected aspects of the book's informatics-related contents, the authors discuss the broader context in which Dr. Lindberg and the NLM accomplished their transformative work.

Commonly used surrogates for baseline renal function affect the classification and prognosis of acute kidney injury.

Studies of acute kidney injury usually lack data on pre-admission kidney function and often substitute an inpatient or imputed serum creatinine as an estimate for baseline renal function. In this study, we compared the potential error introduced by using surrogates such as (1) an estimated glomerular filtration rate of 75 ml/min per 1.73 m(2) (suggested by the Acute Dialysis Quality Initiative), (2) a minimum inpatient serum creatinine value, and (3) the first admission serum creatinine value, with values computed using pre-admission renal function. The study covered a 12-month period and included a cohort of 4863 adults admitted to the Vanderbilt University Hospital. Use of both imputed and minimum baseline serum creatinine values significantly inflated the incidence of acute kidney injury by about half, producing low specificities of 77-80%. In contrast, use of the admission serum creatinine value as baseline significantly underestimated the incidence by about a third, yielding a low sensitivity of 39%. Application of any surrogate marker led to frequent misclassification of patient deaths after acute kidney injury and differences in both in-hospital and 60-day mortality rates. Our study found that commonly used surrogates for baseline serum creatinine result in bi-directional misclassification of the incidence and prognosis of acute kidney injury in a hospital setting.

Computer-assisted diagnostic decision support: history, challenges, and possible paths forward.

This paper presents a brief history of computer-assisted diagnosis, including challenges and future directions. Some ideas presented in this article on computer-assisted diagnostic decision support systems (CDDSS) derive from prior work by the author and his colleagues (see list in Acknowledgments) on the INTERNIST-1 and QMR projects. References indicate the original sources of many of these ideas.

A model for evaluating interface terminologies.

OBJECTIVE: Evaluations of individual terminology systems should be driven in part by the intended usages of such systems. Clinical interface terminologies support interactions between healthcare providers and computer-based applications. They aid practitioners in converting clinical "free text" thoughts into the structured, formal data representations used internally by application programs. Interface terminologies also serve the important role of presenting existing stored, encoded data to end users in human-understandable and actionable formats. The authors present a model for evaluating functional utility of interface terminologies based on these intended uses. DESIGN: Specific parameters defined in the manuscript comprise the metrics for the evaluation model. MEASUREMENTS: Parameters include concept accuracy, term expressivity, degree of semantic consistency for term construction and selection, adequacy of assertional knowledge supporting concepts, degree of complexity of pre-coordinated concepts, and the "human readability" of the terminology. The fundamental metric is how well the interface terminology performs in supporting correct, complete, and efficient data encoding or review by humans. RESULTS: Authors provide examples demonstrating performance of the proposed evaluation model in selected instances. CONCLUSION: A formal evaluation model will permit investigators to evaluate interface terminologies using a consistent and principled approach. Terminology developers and evaluators can apply the proposed model to identify areas for improving interface terminologies.

A computerized insulin infusion titration protocol improves glucose control with less hypoglycemia compared to a manual titration protocol in a trauma intensive care unit.

BACKGROUND: Previous studies reflect reduced morbidity and mortality with intensive blood glucose control in critically ill patients. Unfortunately, implementation of such protocols has proved challenging. This study evaluated the degree of glucose control using manual paper-based vs computer-based insulin protocols in a trauma intensive care unit. METHODS: Of 1455 trauma admissions from May 31 to December 31, 2005, a cohort of 552 critically ill patients met study entry criteria. The patients received intensive blood glucose management with IV insulin infusions. Using Fisher's exact test, the authors compared patients managed with a computerized protocol vs a paper-based insulin protocol with respect to the portion of glucose values in a target range of 80-110 mg/dL, the incidence of hyperglycemia (> or =150 mg/dL), and the incidence of hypoglycemia (< or =40 mg/dL). RESULTS: Three hundred nine patients were managed with a manual paper-based protocol and 243 were managed with a computerized protocol. The total number of blood glucose values across both groups was 21,178. Mean admission glucose was higher in the computer-based protocol group (170 vs 152 mg/dL; p < .001, t-test). Despite this finding by Fisher's exact test, glucose control was superior in the computerized group; a higher portion of glucose values was in range 80-110 mg/dL (41.8% vs 34.0%; p < .001), less hyperglycemia occurred (12.8% vs 15.1%; p < .001), and less hypoglycemia occurred (0.2% vs 0.5%; p < .001). CONCLUSIONS: A computerized insulin titration protocol improves glucose control by (1) increasing the percentage of glucose values in range, (2) reducing hyperglycemia, and (3) reducing severe hypoglycemia.

Evaluation of a Novel System to Enhance Clinicians' Recognition of Preadmission Adverse Drug Reactions.

BACKGROUND: Often unrecognized by providers, adverse drug reactions (ADRs) diminish patients' quality of life, cause preventable admissions and emergency department visits, and increase health care costs. OBJECTIVE: This article evaluates whether an automated system, the Adverse Drug Effect Recognizer (ADER), could assist clinicians in detecting and addressing inpatients' ongoing preadmission ADRs. METHODS: ADER uses natural language processing to extract patients' medications, findings, and past diagnoses from admission notes. It compares excerpted information to a database of known medication adverse effects and promptly warns clinicians about potential ongoing ADRs and potential confounders via alerts placed in patients' electronic health records (EHRs). A 3-month intervention trial evaluated ADER's impact on antihypertensive medication ordering behaviors. At the time of patient admission, ADER warned providers on the Internal Medicine wards of Vanderbilt University Hospital about potential ongoing preadmission antihypertensive medication ADRs. A retrospective control group, comprised similar physicians from a period prior to the intervention, received no alerts. The evaluation compared ordering behaviors for each group to determine if preadmission medications changed during hospitalization or at discharge. The study also analyzed intervention group participants' survey responses and user comments. RESULTS: ADER identified potential preadmission ADRs for 30% of both groups. Compared with controls, intervention providers more often withheld or discontinued suspected ADR-causing medications during the inpatient stay (p < 0.001). Intervention providers who responded to alert-related surveys held or discontinued suspected ADR-causing medications more often at discharge (p < 0.001). CONCLUSION: Results indicate that ADER helped physicians recognize ADRs and reduced ordering of suspected ADR-causing medications. In hospitals using EHRs, ADER-like systems could improve clinicians' recognition and elimination of ongoing ADRs.

Medication administration discrepancies persist despite electronic ordering.

Background Up to 38% of inpatient medication errors occur at the administration stage. Although they reduce prescribing errors, computerized provider order entry (CPOE) systems do not prevent administration errors or timing discrepancies. This study determined the degree to which CPOE medication orders matched actual dose administration times. METHODS At a 658-bed academic hospital with CPOE but lacking electronic medication administration charting, authors randomly selected adult patients with eligible medication orders from historical 1999-2003 CPOE log files. Retrospective manual chart audits compared expected (from CPOE) and actual timing of medication administrations. Outcomes included: dose omissions, median lag times between ordered and charted administrations, unauthorized doses, wrong dose errors, and the rate of nurses' medication schedule shifting. RESULTS Dose omissions occurred in 756 of 6019 (12.6%) audited administration opportunities; only 313 of the omissions (5.2% of opportunities) were unexplained. Wrong doses and unexpected doses occurred for 0.1% and 0.7% of opportunities, respectively. Median lag from expected first dose to actual charted administration time was 27 minutes (IQR 0-127). Nursing staff shifted from ordered to alternate administration schedules for 10.7% of regularly scheduled recurring medication orders. Chart review identified reasons for dose omissions, delays, and dose shifting. CONCLUSION Inpatient CPOE orders are legible and conveyed electronically to nurses and the pharmacy. Nonetheless, ward-based medication administrations do not consistently occur as ordered. Medication administration discrepancies are likely to persist even after implementing CPOE and bar-coded medication administration unless recommended interventions are made to address issues such as determining the true urgency of medication administration, avoiding overlapping duplicative medication orders, and developing a safe means for shifting dosing schedules.

Computer-based insulin infusion protocol improves glycemia control over manual protocol.

OBJECTIVE: Hyperglycemia worsens clinical outcomes in critically ill patients. Precise glycemia control using intravenous insulin improves outcomes. To determine if we could improve glycemia control over a previous paper-based, manual protocol, authors implemented, in a surgical intensive care unit (SICU), an intravenous insulin protocol integrated into a care provider order entry (CPOE) system. DESIGN: Retrospective before-after study of consecutive adult patients admitted to a SICU during pre (manual protocol, 32 days) and post (computer-based protocol, 49 days) periods. MEASUREMENTS: Percentage of glucose readings in ideal range of 70-109 mg/dl, and minutes spent in ideal range of control during the first 5 days of SICU stay. RESULTS: The computer-based protocol reduced time from first glucose measurement to initiation of insulin protocol, improved the percentage of all SICU glucose readings in the ideal range, and improved control in patients on IV insulin for > or =24 hours. Hypoglycemia (<40 mg/dl) was rare in both groups. CONCLUSION: The CPOE-based intravenous insulin protocol improved glycemia control in SICU patients compared to a previous manual protocol, and reduced time to insulin therapy initiation. Integrating a computer-based insulin protocol into a CPOE system achieved efficient, safe, and effective glycemia control in SICU patients.

Physicians' response to guided geriatric dosing: initial results from a randomized trial.

Guided dosing within a computerized provider order entry (CPOE) system is an effective method of individualizing therapy for patients. Physicians' responses to guided dosing decision support have not been extensively studied. As part of a randomized trial evaluating efficacy of dosing advice on reducing falls in the elderly, CPOE prompts to physicians for 88 drugs included tailored messages and guided dose lists with recommended initial doses and frequencies. The study captured all prescribing activity electronically. The primary outcome was the ratio between prescribed dose and recommended dose. Over 9 months, 778 providers entered 9111 study-related medication orders on 2981 patients. Physicians using guided orders chose recommended doses more often than controls(28.6% vs. 24.1%, p<0.001). Selected doses were significantly lower in the intervention group (median ratio of actual to recommended 2.5, interquartile range [1.0,4.0]) than the control group (median 3.0 interquartile range [1.5,5.0], p<0.001). While physicians selected the recommended dose less than a third of the time, guided geriatric dosing modestly improved compliance with guidelines.