SpotSDC: Revealing the Silent Data Corruption Propagation in High-Performance Computing Systems.

The trend of rapid technology scaling is expected to make the hardware of high-performance computing (HPC) systems more susceptible to computational errors due to random bit flips. Some bit flips may cause a program to crash or have a minimal effect on the output, but others may lead to silent data corruption (SDC), i.e., undetected yet significant output errors. Classical fault injection analysis methods employ uniform sampling of random bit flips during program execution to derive a statistical resiliency profile. However, summarizing such fault injection result with sufficient detail is difficult, and understanding the behavior of the fault-corrupted program is still a challenge. In this article, we introduce SpotSDC, a visualization system to facilitate the analysis of a program's resilience to SDC. SpotSDC provides multiple perspectives at various levels of detail of the impact on the output relative to where in the source code the flipped bit occurs, which bit is flipped, and when during the execution it happens. SpotSDC also enables users to study the code protection and provide new insights to understand the behavior of a fault-injected program. Based on lessons learned, we demonstrate how what we found can improve the fault injection campaign method.

Visual Exploration of Semantic Relationships in Neural Word Embeddings.

Constructing distributed representations for words through neural language models and using the resulting vector spaces for analysis has become a crucial component of natural language processing (NLP). However, despite their widespread application, little is known about the structure and properties of these spaces. To gain insights into the relationship between words, the NLP community has begun to adapt high-dimensional visualization techniques. In particular, researchers commonly use t-distributed stochastic neighbor embeddings (t-SNE) and principal component analysis (PCA) to create two-dimensional embeddings for assessing the overall structure and exploring linear relationships (e.g., word analogies), respectively. Unfortunately, these techniques often produce mediocre or even misleading results and cannot address domain-specific visualization challenges that are crucial for understanding semantic relationships in word embeddings. Here, we introduce new embedding techniques for visualizing semantic and syntactic analogies, and the corresponding tests to determine whether the resulting views capture salient structures. Additionally, we introduce two novel views for a comprehensive study of analogy relationships. Finally, we augment t-SNE embeddings to convey uncertainty information in order to allow a reliable interpretation. Combined, the different views address a number of domain-specific tasks difficult to solve with existing tools.

Interactive Visualization and Exploration of Patient Progression in a Hospital Setting.

As medical organizations increasingly adopt the use of electronic health records (EHRs), large volumes of clinical data are being captured on a daily basis. These data provide comprehensive information about patients and have the potential to improve a wide range of application domains in healthcare. Physicians and clinical researchers are interested in finding effective ways to understand this abundance of data. Use of visual analytics to explore healthcare data is one such research direction. Here, we present a visualization and analysis environment to understand patient progression over time. Through the use of optimized data structures and progressive visualization techniques, we allow users to interactively explore how patients and their progression change over time. Compared to existing techniques, our work provides additional flexibility in analyzing patient data and has the potential to be used in a real-time hospital setting. Finally, we demonstrate the utility of our approach using a publicly available intensive care unit (ICU) database.

Big-Data Based Decision-Support Systems to Improve Clinicians' Cognition.

Complex clinical decision-making could be facilitated by using population health data to inform clinicians. In two previous studies, we interviewed 16 infectious disease experts to understand complex clinical reasoning. For this study, we focused on answers from the experts on how clinical reasoning can be supported by population-based Big-Data. We found cognitive strategies such as trajectory tracking, perspective taking, and metacognition has the potential to improve clinicians' cognition to deal with complex problems. These cognitive strategies could be supported by population health data, and all have important implications for the design of Big-Data based decision-support tools that could be embedded in electronic health records. Our findings provide directions for task allocation and design of decision-support applications for health care industry development of Big data based decision-support systems.

Evaluation of an Electronic Module for Reconciling Medications in Home Health Plans of Care.

OBJECTIVES: Transitions in patient care pose an increased risk to patient safety. One way to reduce this risk is to ensure accurate medication reconciliation during the transition. Here we present an evaluation of an electronic medication reconciliation module we developed to reduce the transition risk in patients referred for home healthcare. METHODS: Nineteen physicians with experience in managing home health referrals were recruited to participate in this within-subjects experiment. Participants completed medication reconciliation for three clinical cases in each of two conditions. The first condition (paper-based) simulated current practice - reconciling medication discrepancies between a paper plan of care (CMS 485) and a simulated Electronic Health Record (EHR). For the second condition (electronic) participants used our medication reconciliation module, which we integrated into the simulated EHR. To evaluate the effectiveness of our medication reconciliation module, we employed repeated measures ANOVA to test the hypotheses that the module will: 1) Improve accuracy by reducing the number of unaddressed medication discrepancies, 2) Improve efficiency by reducing the reconciliation time, 3) have good perceived usability. RESULTS: The improved accuracy hypothesis is supported. Participants left more discrepancies unaddressed in the paper-based condition than the electronic condition, F (1,1) = 22.3, p < 0.0001 (Paper Mean = 1.55, SD = 1.20; Electronic Mean = 0.45, SD = 0.65). However, contrary to our efficiency hypothesis, participants took the same amount of time to complete cases in the two conditions, F (1, 1) =0.007, p = 0.93 (Paper Mean = 258.7 seconds, SD = 124.4; Electronic Mean = 260.4 seconds, SD = 158.9). The usability hypothesis is supported by a composite mean ability and confidence score of 6.41 on a 7-point scale, 17 of 19 participants preferring the electronic system and an SUS rating of 86.5. CONCLUSION: We present the evaluation of an electronic medication reconciliation module that increases detection and resolution of medication discrepancies compared to a paper-based process. Further work to integrate medication reconciliation within an electronic medical record is warranted.

Feasibility of Population Health Analytics and Data Visualization for Decision Support in the Infectious Diseases Domain: A pilot study.

OBJECTIVE: Big data or population-based information has the potential to reduce uncertainty in medicine by informing clinicians about individual patient care. The objectives of this study were: 1) to explore the feasibility of extracting and displaying population-based information from an actual clinical population's database records, 2) to explore specific design features for improving population display, 3) to explore perceptions of population information displays, and 4) to explore the impact of population information display on cognitive outcomes. METHODS: We used the Veteran's Affairs (VA) database to identify similar complex patients based on a similar complex patient case. Study outcomes measures were 1) preferences for population information display 2) time looking at the population display, 3) time to read the chart, and 4) appropriateness of plans with pre- and post-presentation of population data. Finally, we redesigned the population information display based on our findings from this study. RESULTS: The qualitative data analysis for preferences of population information display resulted in four themes: 1) trusting the big/population data can be an issue, 2) embedded analytics is necessary to explore patient similarities, 3) need for tools to control the view (overview, zoom and filter), and 4) different presentations of the population display can be beneficial to improve the display. We found that appropriateness of plans was at 60% for both groups (t9=-1.9; p=0.08), and overall time looking at the population information display was 2.3 minutes versus 3.6 minutes with experts processing information faster than non-experts (t8= -2.3, p=0.04). CONCLUSION: A population database has great potential for reducing complexity and uncertainty in medicine to improve clinical care. The preferences identified for the population information display will guide future health information technology system designers for better and more intuitive display.

Automated concept and relationship extraction for the semi-automated ontology management (SEAM) system.

BACKGROUND: We develop medical-specialty specific ontologies that contain the settled science and common term usage. We leverage current practices in information and relationship extraction to streamline the ontology development process. Our system combines different text types with information and relationship extraction techniques in a low overhead modifiable system. Our SEmi-Automated ontology Maintenance (SEAM) system features a natural language processing pipeline for information extraction. Synonym and hierarchical groups are identified using corpus-based semantics and lexico-syntactic patterns. The semantic vectors we use are term frequency by inverse document frequency and context vectors. Clinical documents contain the terms we want in an ontology. They also contain idiosyncratic usage and are unlikely to contain the linguistic constructs associated with synonym and hierarchy identification. By including both clinical and biomedical texts, SEAM can recommend terms from those appearing in both document types. The set of recommended terms is then used to filter the synonyms and hierarchical relationships extracted from the biomedical corpus. We demonstrate the generality of the system across three use cases: ontologies for acute changes in mental status, Medically Unexplained Syndromes, and echocardiogram summary statements. RESULTS: Across the three uses cases, we held the number of recommended terms relatively constant by changing SEAM's parameters. Experts seem to find more than 300 recommended terms to be overwhelming. The approval rate of recommended terms increased as the number and specificity of clinical documents in the corpus increased. It was 60% when there were 199 clinical documents that were not specific to the ontology domain and 90% when there were 2879 documents very specific to the target domain. We found that fewer than 100 recommended synonym groups were also preferred. Approval rates for synonym recommendations remained low varying from 43% to 25% as the number of journal articles increased from 19 to 47. Overall the number of recommended hierarchical relationships was very low although approval was good. It varied between 67% and 31%. CONCLUSION: SEAM produced a concise list of recommended clinical terms, synonyms and hierarchical relationships regardless of medical domain.

The EpiCanvas infectious disease weather map: an interactive visual exploration of temporal and spatial correlations.

Advances in surveillance science have supported public health agencies in tracking and responding to disease outbreaks. Increasingly, epidemiologists have been tasked with interpreting multiple streams of heterogeneous data arising from varied surveillance systems. As a result public health personnel have experienced an overload of plots and charts as information visualization techniques have not kept pace with the rapid expansion in data availability. This study sought to advance the science of public health surveillance data visualization by conceptualizing a visual paradigm that provides an 'epidemiological canvas' for detection, monitoring, exploration and discovery of regional infectious disease activity and developing a software prototype of an 'infectious disease weather map'. Design objectives were elucidated and the conceptual model was developed using cognitive task analysis with public health epidemiologists. The software prototype was pilot tested using retrospective data from a large, regional pediatric hospital, and gastrointestinal and respiratory disease outbreaks were re-created as a proof of concept.

Epinome: a visual-analytics workbench for epidemiology data.

Early detection and rapid response to infectious-disease outbreaks rely on effective decision making based on information from disparate sources. To improve decision-making in outbreak detection and response, it's important to understand how public health practitioners seek relevant information. Epinome, a user-centric visual-analytics system, supports research on decision-making in public health, particularly evaluation of information search strategies. Epinome facilitates investigation of scripted high-fidelity large-scale simulated disease outbreaks. Its dynamic environment seamlessly evolves and adapts as the user's tasks and focus change. This video shows how the Epinome system facilitates interactive simulations of disease outbreaks.

Strengthening Partnerships along the Informatics Innovation Stages and Spaces: Research and Practice Collaboration in Utah.

Collaborate, translate, and impact are key concepts describing the roles and purposes of the research Centers of Excellence (COE) in Public Health Informatics (PHI). Rocky Mountain COE integrated these concepts into a framework of PHI Innovation Space and Stage to guide their collaboration between the University of Utah, Intermountain Healthcare, and Utah Department of Health. Seven research projects are introduced that illustrate the framework and demonstrate how to effectively manage multiple innovations among multiple organizations over a five-year period. A COE is more than an aggregation of distinct research projects over a short time period. The people, partnership, shared vision, and mutual understanding and appreciation developed over a long period of time form the core and foundation for ongoing collaborative innovations and its successes.

Epinome - a novel workbench for epidemic investigation and analysis of search strategies in public health practice.

We present a novel user-centric visual analytics system that supports investigation of simulated disease outbreak and the study of decision-making. We developed Epinome as part of our research on decision making in public health and in particular, on the evaluation of information search strategies in public health practice. Epinome is a highly dynamic web-based system that provides a platform to track and study subjects' decision making and information search strategies, under controlled and repeatable conditions using simulated disease outbreaks. In this paper we focus on the design and implementation of Epinome and present relevant results from field tests we conducted in Utah and Colorado.

A survey of radial methods for information visualization.

Radial visualization, or the practice of displaying data in a circular or elliptical pattern, is an increasingly common technique in information visualization research. In spite of its prevalence, little work has been done to study this visualization paradigm as a methodology in its own right. We provide a historical review of radial visualization, tracing it to its roots in centuries-old statistical graphics. We then identify the types of problem domains to which modern radial visualization techniques have been applied. A taxonomy for radial visualization is proposed in the form of seven design patterns encompassing nearly all recent works in this area. From an analysis of these patterns, we distill a series of design considerations that system builders can use to create new visualizations that address aspects of the design space that have not yet been explored. It is hoped that our taxonomy will provide a framework for facilitating discourse among researchers and stimulate the development of additional theories and systems involving radial visualization as a distinct design metaphor.