Why is biomedical informatics hard? A fundamental framework.

Building on previous work to define the scientific discipline of biomedical informatics, we present a framework that categorizes fundamental challenges into groups based on data, information, and knowledge, along with the transitions between these levels. We define each level and argue that the framework provides a basis for separating informatics problems from non-informatics problems, identifying fundamental challenges in biomedical informatics, and provides guidance regarding the search for general, reusable solutions to informatics problems. We distinguish between processing data (symbols) and processing meaning. Computational systems, that are the basis for modern information technology (IT), process data. In contrast, many important challenges in biomedicine, such as providing clinical decision support, require processing meaning, not data. Biomedical informatics is hard because of the fundamental mismatch between many biomedical problems and the capabilities of current technology.

Assessing the completeness of periodontal disease documentation in the EHR: a first step in measuring the quality of care.

BACKGROUND: Our objective was to measure the proportion of patients for which comprehensive periodontal charting, periodontal disease risk factors (diabetes status, tobacco use, and oral home care compliance), and periodontal diagnoses were documented in the electronic health record (EHR). We developed an EHR-based quality measure to assess how well four dental institutions documented periodontal disease-related information. An automated database script was developed and implemented in the EHR at each institution. The measure was validated by comparing the findings from the measure with a manual review of charts. RESULTS: The overall measure scores varied significantly across the four institutions (institution 1 = 20.47%, institution 2 = 0.97%, institution 3 = 22.27% institution 4 = 99.49%, p-value < 0.0001). The largest gaps in documentation were related to periodontal diagnoses and capturing oral homecare compliance. A random sample of 1224 charts were manually reviewed and showed excellent validity when compared with the data generated from the EHR-based measure (Sensitivity, Specificity, PPV, and NPV > 80%). CONCLUSION: Our results demonstrate the feasibility of developing automated data extraction scripts using structured data from EHRs, and successfully implementing these to identify and measure the periodontal documentation completeness within and across different dental institutions.

Dashboard visualizations: Supporting real-time throughput decision-making.

Providing timely and effective care in the emergency department (ED) requires the management of individual patients as well as the flow and demands of the entire department. Strategic changes to work processes, such as adding a flow coordination nurse or a physician in triage, have demonstrated improvements in throughput times. However, such global strategic changes do not address the real-time, often opportunistic workflow decisions of individual clinicians in the ED. We believe that real-time representation of the status of the entire emergency department and each patient within it through information visualizations will better support clinical decision-making in-the-moment and provide for rapid intervention to improve ED flow. This notion is based on previous work where we found that clinicians' workflow decisions were often based on an in-the-moment local perspective, rather than a global perspective. Here, we discuss the challenges of designing and implementing visualizations for ED through a discussion of the development of our prototype Throughput Dashboard and the potential it holds for supporting real-time decision-making.

Improving the utility of MeSH® terms using the TopicalMeSH representation.

OBJECTIVE: To evaluate whether vector representations encoding latent topic proportions that capture similarities to MeSH terms can improve performance on biomedical document retrieval and classification tasks, compared to using MeSH terms. MATERIALS AND METHODS: We developed the TopicalMeSH representation, which exploits the 'correspondence' between topics generated using latent Dirichlet allocation (LDA) and MeSH terms to create new document representations that combine MeSH terms and latent topic vectors. We used 15 systematic drug review corpora to evaluate performance on information retrieval and classification tasks using this TopicalMeSH representation, compared to using standard encodings that rely on either (1) the original MeSH terms, (2) the text, or (3) their combination. For the document retrieval task, we compared the precision and recall achieved by ranking citations using MeSH and TopicalMeSH representations, respectively. For the classification task, we considered three supervised machine learning approaches, Support Vector Machines (SVMs), logistic regression, and decision trees. We used these to classify documents as relevant or irrelevant using (independently) MeSH, TopicalMeSH, Words (i.e., n-grams extracted from citation titles and abstracts, encoded via bag-of-words representation), a combination of MeSH and Words, and a combination of TopicalMeSH and Words. We also used SVM to compare the classification performance of tf-idf weighted MeSH terms, LDA Topics, a combination of Topics and MeSH, and TopicalMeSH to supervised LDA's classification performance. RESULTS: For the document retrieval task, using the TopicalMeSH representation resulted in higher precision than MeSH in 11 of 15 corpora while achieving the same recall. For the classification task, use of TopicalMeSH features realized a higher F1 score in 14 of 15 corpora when used by SVMs, 12 of 15 corpora using logistic regression, and 12 of 15 corpora using decision trees. TopicalMeSH also had better document classification performance on 12 of 15 corpora when compared to Topics, tf-idf weighted MeSH terms, and a combination of Topics and MeSH using SVMs. Supervised LDA achieved the worst performance in most of the corpora. CONCLUSION: The proposed TopicalMeSH representation (which combines MeSH terms with latent topics) consistently improved performance on document retrieval and classification tasks, compared to using alternative standard representations using MeSH terms alone, as well as, several standard alternative approaches.

SYFSA: a framework for systematic yet flexible systems analysis.

Although technological or organizational systems that enforce systematic procedures and best practices can lead to improvements in quality, these systems must also be designed to allow users to adapt to the inherent uncertainty, complexity, and variations in healthcare. We present a framework, called Systematic Yet Flexible Systems Analysis (SYFSA) that supports the design and analysis of Systematic Yet Flexible (SYF) systems (whether organizational or technical) by formally considering the tradeoffs between systematicity and flexibility. SYFSA is based on analyzing a task using three related problem spaces: the idealized space, the natural space, and the system space. The idealized space represents the best practice-how the task is to be accomplished under ideal conditions. The natural space captures the task actions and constraints on how the task is currently done. The system space specifies how the task is done in a redesigned system, including how it may deviate from the idealized space, and how the system supports or enforces task constraints. The goal of the framework is to support the design of systems that allow graceful degradation from the idealized space to the natural space. We demonstrate the application of SYFSA for the analysis of a simplified central line insertion task. We also describe several information-theoretic measures of flexibility that can be used to compare alternative designs, and to measure how efficiently a system supports a given task, the relative cognitive workload, and learnability.

Evaluation of a problem-specific SBAR tool to improve after-hours nurse-physician phone communication: a randomized trial.

BACKGROUND: After-hours telephone communications are common in patient management. Patterns of communication of key information during after-hours phone calls were evaluated, and the utility of problem-specific Situation, Background, Assessment, Recommendation (SBAR) forms in improving this communication was assessed. METHODS: In a randomized trial using a simulated on-call setting, 20 nurses called physicians regarding six cases adapted from inpatient records and based on the six most common reasons for after-hours nurse-physician communication. Three of the cases were handled without the SBAR forms (control cases), and three cases were handled with the forms (SBAR cases). Two cue types of communication were evaluated: situation cues, which conveyed the patient's situation (for example, a patient is confused), and background cues, which conveyed problem-specific data indicated on the SBAR forms (for example, the patient has a low sodium level). RESULTS: Ninety-two phone calls were analyzed (43 SBAR/49 controls). Most of the nurses reported the situation cues (SBAR 88%, control 84%, p = .60) but not the background cues. There was a trend toward fewer background cues communicated in the SBAR cases (14% versus 31%, p = .08). In 14% of the cases, on average, nurses omitted information or reported wrong information regarding the background cue. Physicians asked questions that resulted in the communication of the cues in a minority of the cases when the background cues were not originally provided by the nurses (SBAR 6%, control 16%, p = .39). CONCLUSIONS: In after-hours phone communication between physicians and nurses, significant information was often not communicated and physicians did not elicit the necessary information. Simply providing an SBAR-based form did not ensure complete communication of key information.

Mapping the delineation of practice to the AMIA foundational domains for applied health informatics.

OBJECTIVE: This article reports on the alignment between the foundational domains and the delineation of practice (DoP) for health informatics, both developed by the American Medical Informatics Association (AMIA). Whereas the foundational domains guide graduate-level curriculum development and accreditation assessment, providing an educational pathway to the minimum competencies needed as a health informatician, the DoP defines the domains, tasks, knowledge, and skills that a professional needs to competently perform in the discipline of health informatics. The purpose of this article is to determine whether the foundational domains need modification to better reflect applied practice. MATERIALS AND METHODS: Using an iterative process and through individual and collective approaches, the foundational domains and the DoP statements were analyzed for alignment and eventual harmonization. Tables and Sankey plot diagrams were used to detail and illustrate the resulting alignment. RESULTS: We were able to map all the individual DoP knowledge statements and tasks to the AMIA foundational domains, but the statements within a single DoP domain did not all map to the same foundational domain. Even though the AMIA foundational domains and DoP domains are not in perfect alignment, the DoP provides good examples of specific health informatics competencies for most of the foundational domains. There are, however, limited DoP knowledge statements and tasks mapping to foundational domain 6-Social and Behavioral Aspects of Health. DISCUSSION: Both the foundational domains and the DoP were developed independently, several years apart, and for different purposes. The mapping analyses reveal similarities and differences between the practice experience and the curricular needs of health informaticians. CONCLUSIONS: The overall alignment of both domains may be explained by the fact that both describe the current and/or future health informatics professional. One can think of the foundational domains as representing the broad foci for educational programs for health informaticians and, hence, they are appropriately the focus of organizations that accredit these programs.

Cross-terminology mapping challenges: a demonstration using medication terminological systems.

Standardized terminological systems for biomedical information have provided considerable benefits to biomedical applications and research. However, practical use of this information often requires mapping across terminological systems-a complex and time-consuming process. This paper demonstrates the complexity and challenges of mapping across terminological systems in the context of medication information. It provides a review of medication terminological systems and their linkages, then describes a case study in which we mapped proprietary medication codes from an electronic health record to SNOMED CT and the UMLS Metathesaurus. The goal was to create a polyhierarchical classification system for querying an i2b2 clinical data warehouse. We found that three methods were required to accurately map the majority of actively prescribed medications. Only 62.5% of source medication codes could be mapped automatically. The remaining codes were mapped using a combination of semi-automated string comparison with expert selection, and a completely manual approach. Compound drugs were especially difficult to map: only 7.5% could be mapped using the automatic method. General challenges to mapping across terminological systems include (1) the availability of up-to-date information to assess the suitability of a given terminological system for a particular use case, and to assess the quality and completeness of cross-terminology links; (2) the difficulty of correctly using complex, rapidly evolving, modern terminologies; (3) the time and effort required to complete and evaluate the mapping; (4) the need to address differences in granularity between the source and target terminologies; and (5) the need to continuously update the mapping as terminological systems evolve.

Caries Risk Documentation And Prevention: eMeasures For Dental Electronic Health Records.

BACKGROUND: Longitudinal patient level data available in the electronic health record (EHR) allows for the development, implementation, and validations of dental quality measures (eMeasures). OBJECTIVE: We report the feasibility and validity of implementing two eMeasures. The eMeasures determined the proportion of patients receiving a caries risk assessment (eCRA) and corresponding appropriate risk-based preventative treatments for patients at elevated risk of caries (appropriateness of care [eAoC]) in two academic institutions and one accountable care organization, in the 2019 reporting year. METHODS: Both eMeasures define the numerator and denominator beginning at the patient level, populations' specifications, and validated the automated queries. For eCRA, patients who completed a comprehensive or periodic oral evaluation formed the denominator, and patients of any age who received a CRA formed the numerator. The eAoC evaluated the proportion of patients at elevated caries risk who received the corresponding appropriate risk-based preventative treatments. RESULTS: EHR automated queries identified in three sites 269,536 patients who met the inclusion criteria for receiving a CRA. The overall proportion of patients who received a CRA was 94.4% (eCRA). In eAoC, patients at elevated caries risk levels (moderate, high, or extreme) received fluoride preventive treatment ranging from 56 to 93.8%. For patients at high and extreme risk, antimicrobials were prescribed more frequently site 3 (80.6%) than sites 2 (16.7%) and 1 (2.9%). CONCLUSION: Patient-level data available in the EHRs can be used to implement process-of-care dental eCRA and AoC, eAoC measures identify gaps in clinical practice. EHR-based measures can be useful in improving delivery of evidence-based preventative treatments to reduce risk, prevent tooth decay, and improve oral health.

Opportunistic decision making and complexity in emergency care.

In critical care environments such as the emergency department (ED), many activities and decisions are not planned. In this study, we developed a new methodology for systematically studying what are these unplanned activities and decisions. This methodology expands the traditional naturalistic decision making (NDM) frameworks by explicitly identifying the role of environmental factors in decision making. We focused on decisions made by ED physicians as they transitioned between tasks. Through ethnographic data collection, we developed a taxonomy of decision types. The empirical data provide important insight to the complexity of the ED environment by highlighting adaptive behavior in this intricate milieu. Our results show that half of decisions in the ED we studied are not planned, rather decisions are opportunistic decision (34%) or influenced by interruptions or distractions (21%). What impacts these unplanned decisions have on the quality, safety, and efficiency in the ED environment are important research topics for future investigation.

Exploring the Hazards of Scaling Up Clinical Data Analyses: A Drug Side Effect Discovery Case Report.

We assessed the scalability of pharmacological signal detection use case from a single-site CDW to a large aggregated clinical data warehouse (single-site database with 754,214 distinct patient IDs vs. multisite database with 49.8M). We aimed to explore whether a larger clinical dataset would provide clearer signals for secondary analyses such as detecting the known relationship between prednisone and weight. We found significant weight gain rate using the single-site data but not from using aggregated data (0.0104 kg/day, p<0.0001 vs. -0.050 kg/day, p<.0001). This rate was also found more consistently across 30 age and gender subgroups using the single-site data than in the aggregated data (26 vs. 18 significant weight gain findings). Contrary to our expectations, analyses of much larger aggregated clinical datasets did not yield stronger signals. Researchers must check the underlying model assumptions and account for greater heterogeneity when analyzing aggregated multisite data to ensure reliable findings.

What is biomedical informatics?

Biomedical informatics lacks a clear and theoretically-grounded definition. Many proposed definitions focus on data, information, and knowledge, but do not provide an adequate definition of these terms. Leveraging insights from the philosophy of information, we define informatics as the science of information, where information is data plus meaning. Biomedical informatics is the science of information as applied to or studied in the context of biomedicine. Defining the object of study of informatics as data plus meaning clearly distinguishes the field from related fields, such as computer science, statistics and biomedicine, which have different objects of study. The emphasis on data plus meaning also suggests that biomedical informatics problems tend to be difficult when they deal with concepts that are hard to capture using formal, computational definitions. In other words, problems where meaning must be considered are more difficult than problems where manipulating data without regard for meaning is sufficient. Furthermore, the definition implies that informatics research, teaching, and service should focus on biomedical information as data plus meaning rather than only computer applications in biomedicine.

Measuring sealant placement in children at the dental practice level.

BACKGROUND: Although sealants are an established and recommended caries-preventive treatment, many children still fail to receive them. In addition, research has shown that existing measures underestimate care by overlooking the sealable potential of teeth before evaluating care. To address this, the authors designed and evaluated 3 novel dental electronic health record-based clinical quality measures that evaluate sealant care only after assessing the sealable potential of teeth. METHODS: Measure I recorded the proportion of patients with sealable teeth who received sealants. Measure II recorded the proportion of patients who had at least 1 of their sealable teeth sealed. Measure III recorded the proportion of patients who received sealant on all of their sealable teeth. RESULTS: On average, 48.1% of 6- through 9-year-old children received 1 or more sealants compared with 32.4% of 10- through 14-year-olds (measure I). The average measure score decreased for patients who received sealants for at least 1 of their sealable teeth (measure II) (43.2% for 6- through 9-year-olds and 28.4% for 10- through 14-year-olds). Fewer children received sealants on all eligible teeth (measure III) (35.5% of 6- through 9-year-olds and 21% of 10- through 14-year-olds received sealant on all eligible teeth). Among the 48.5% who were at elevated caries risk, the sealant rates were higher across all 3 measures. CONCLUSIONS: A valid and actionable practice-based sealant electronic measure that evaluates sealant treatment among the eligible population, both at the patient level and the tooth level, has been developed. PRACTICAL IMPLICATIONS: The measure developed in this work provides practices with patient-centered and actionable sealant quality measures that aim to improve oral health outcomes.

Legibility of a volumetric infusion pump in a shock trauma ICU.

BACKGROUND: Reports submitted to the Federal Food and Drug Administration (FDA) indicate that legibility of infusion pump interfaces contributes to medication errors (for example, patients have been seriously injured when nurses overinfused them after reading the number 7 as a 1). Health care has experienced an influx of medical devices with small-screen interfaces, generically described as small-screen devices. Legibility is widely acknowledged as a necessary aspect of safe operation of medical devices. Contextual analysis was used to observe conditions affecting the legibility of the screen interface of a dual-channel infusion volumetric infusion pump in a shock trauma intensive care unit (STICU). METHODS: Observations were made of registered nurses and physicians using the infusion pumps during a four-hour period. RESULTS: Results from the observations indicated that there was reduced legibility of the infusion pump screen interface because of an inherently subdued light situation, reduced screen contrast, and the small font size of the lettering. DISCUSSION: It was clear from the beginning of the observations that the screen of the infusion pump had limited legibility. In some instances, nurses attached handmade tape labels to the infusion pump to enhance and supplement the small screen. The pump was often positioned facing away from the nurses, who then had to reposition it to view the screen, contributing to interruptions in work flow and creating a potential safety hazard. A variety of strategies are recommended to ensure legibility of infusion pump interfaces and of medical-device labels. CONCLUSIONS: A more complete environmental approach is needed to determine the legibility and usefulness of microdisplay and small-screen devices in health care.

DataGauge: A Practical Process for Systematically Designing and Implementing Quality Assessments of Repurposed Clinical Data.

The well-known hazards of repurposing data make Data Quality (DQ) assessment a vital step towards ensuring valid results regardless of analytical methods. However, there is no systematic process to implement DQ assessments for secondary uses of clinical data. This paper presents DataGauge, a systematic process for designing and implementing DQ assessments to evaluate repurposed data for a specific secondary use. DataGauge is composed of five steps: (1) Define information needs, (2) Develop a formal Data Needs Model (DNM), (3) Use the DNM and DQ theory to develop goal-specific DQ assessment requirements, (4) Extract DNM-specified data, and (5) Evaluate according to DQ requirements. DataGauge's main contribution is integrating general DQ theory and DQ assessment methods into a systematic process. This process supports the integration and practical implementation of existing Electronic Health Record-specific DQ assessment guidelines. DataGauge also provides an initial theory-based guidance framework that ties the DNM to DQ testing methods for each DQ dimension to aid the design of DQ assessments. This framework can be augmented with existing DQ guidelines to enable systematic assessment. DataGauge sets the stage for future systematic DQ assessment research by defining an assessment process, capable of adapting to a broad range of clinical datasets and secondary uses. Defining DataGauge sets the stage for new research directions such as DQ theory integration, DQ requirements portability research, DQ assessment tool development and DQ assessment tool usability.

Interruptions in a level one trauma center: a case study.

BACKGROUND: The emergency department has been characterized as interrupt-driven. Government agencies and patient safety organizations recognize that interruptions contribute to medical errors. The purpose of this study was to observe, record, and contextualize activities and interruptions experienced by physicians and Registered Nurses (RNs) working in a Level One Trauma Center. DESIGN: A case study that relied on an ethnographic study design using the shadowing method. SUBJECTS: A convenience sample of physicians and RNs, each with at least 6 months of experience in the Emergency Department (ED), were asked to participate. In these kinds of detailed qualitative investigations, it is quite common to have a small sample size. Ethical approval: Approval was obtained from institutional ethic committees prior to initiating the study. Community consent was obtained from the ED staff through in-service education. SETTING: All observations were made in the trauma section of the ED of a tertiary teaching hospital. The hospital is situated in a major medical center in the Gulf Coast region of the United States of America (USA). FINDINGS: Five attending ED physicians were observed for a total of 29h, 31min. Eight RNs were shadowed for a total of 40 h, 9min. Interruptions and activities were categorized using the Hybrid Method to Categorize Interruptions and Activities (HyMCIA). Registered Nurses received slightly more interruptions per hour than physicians. People, pagers, and telephones were identified as mediums through which interruptions were delivered. The physical environment was found to contribute to interruptions in workflow because of physical design and when supplies were not available. Physicians and RNs usually returned to the original, interrupted activity more often than leaving the activity unfinished. CONCLUSION: This research provides an enhanced understanding of interruptions in workflow in the ED, the identification of work constraints, and the need to develop interventions to manage interruptions. It is crucial that interruptions be delivered in such a way that there is minimal negative impact on performance. The significance and importance of the interruption must always be weighed against the negative impact that it has on smooth, efficient workflow.

User needs analysis and usability assessment of DataMed - a biomedical data discovery index.

OBJECTIVE: To present user needs and usability evaluations of DataMed, a Data Discovery Index (DDI) that allows searching for biomedical data from multiple sources. MATERIALS AND METHODS: We conducted 2 phases of user studies. Phase 1 was a user needs analysis conducted before the development of DataMed, consisting of interviews with researchers. Phase 2 involved iterative usability evaluations of DataMed prototypes. We analyzed data qualitatively to document researchers' information and user interface needs. RESULTS: Biomedical researchers' information needs in data discovery are complex, multidimensional, and shaped by their context, domain knowledge, and technical experience. User needs analyses validate the need for a DDI, while usability evaluations of DataMed show that even though aggregating metadata into a common search engine and applying traditional information retrieval tools are promising first steps, there remain challenges for DataMed due to incomplete metadata and the complexity of data discovery. DISCUSSION: Biomedical data poses distinct problems for search when compared to websites or publications. Making data available is not enough to facilitate biomedical data discovery: new retrieval techniques and user interfaces are necessary for dataset exploration. Consistent, complete, and high-quality metadata are vital to enable this process. CONCLUSION: While available data and researchers' information needs are complex and heterogeneous, a successful DDI must meet those needs and fit into the processes of biomedical researchers. Research directions include formalizing researchers' information needs, standardizing overviews of data to facilitate relevance judgments, implementing user interfaces for concept-based searching, and developing evaluation methods for open-ended discovery systems such as DDIs.

Reasons for after-hours calls by hospital floor nurses to on-call physicians.

BACKGROUND: Communication failure is a common root cause of preventable medical errors affecting hospitalized patients. A study was conducted to determine the reasons for calls made by nurses working on the general medical wards to on-call physicians from 6:00 P.M. to 7:00 A.M. METHODS: A retrospective review was performed of a random sample of 500 inpatients admitted to general medical wards at an urban public teaching hospital in Houston between January 1, 2000, and February 28, 2003. RESULTS: In 139 (47%) of 293 medical records there were 304 documented calls. The majority of calls (81%) took place between 6:00 P.M. and 2:00 A.M., with peak call volume between midnight and 2:00 P.M. Patients with one or more calls had an average of 2.2 calls during their stay. Ten categories accounted for 65% of all the nurse calls. In 44% of calls, physicians responded by ordering a medication. DISCUSSION: Communication between floor nurses and on-call physicians might be improved by several interventions. Because 10 reasons accounted for 65% of after-hours calls, protocols could allow nurses to resolve some acute problems without physician involvement. For example, appropriate standing orders (e.g., PR.N. medications) may prevent some calls. In addition, sign-out procedures can be tailored to address common problems that are likely to require future telephone communication. With efforts to change error-prone systems, it seems prudent to focus on after-hours coverage.

Attitudes toward medical device use errors and the prevention of adverse events.

BACKGROUND: The design of a device's user interface often contributes to the chance of a user making an error in using the device. However, there is evidence that most such errors that occur in practice are attributed solely to the user and that the primary method of error prevention is to retrain the user. Yet this attitude may decrease the quality of error reports and the use of more effective error prevention strategies. A qualitative study was conducted to assess health care employees' attitudes toward device use errors and the prevention of adverse events. METHODS: Twenty-six health care employees from three hospital systems, including 11 device users and 15 nonusers who had participated in infusion pump purchasing decisions were given a scenario describing a device use error involving an infusion pump. Several open-ended questions assessed what they felt led to the event and how they would prevent the event from reccurring. RESULTS: The top three reported types of factors leading to the adverse event, in decreasing order of frequency, were the user, pump design problems, and lack of training. The top three prevention strategies reported by the participants were retraining the user, redesigning the device, and telling the user to be careful. DISCUSSION: These results suggest that health care employees still put too much emphasis on the traditional view of blaming and retraining the user.