IVESA - Visual Analysis of Time-Stamped Event Sequences.

Time-stamped event sequences (TSEQs) are time-oriented data without value information, shifting the focus of users to the exploration of temporal event occurrences. TSEQs exist in application domains, such as sleeping behavior, earthquake aftershocks, and stock market crashes. Domain experts face four challenges, for which they could use interactive and visual data analysis methods. First, TSEQs can be large with respect to both the number of sequences and events, often leading to millions of events. Second, domain experts need validated metrics and features to identify interesting patterns. Third, after identifying interesting patterns, domain experts contextualize the patterns to foster sensemaking. Finally, domain experts seek to reduce data complexity by data simplification and machine learning support. We present IVESA, a visual analytics approach for TSEQs. It supports the analysis of TSEQs at the granularities of sequences and events, supported with metrics and feature analysis tools. IVESA has multiple linked views that support overview, sort+filter, comparison, details-on-demand, and metadata relation-seeking tasks, as well as data simplification through feature analysis, interactive clustering, filtering, and motif detection and simplification. We evaluated IVESA with three case studies and a user study with six domain experts working with six different datasets and applications. Results demonstrate the usability and generalizability of IVESA across applications and cases that had up to 1,000,000 events.

Lightning and Thunder: The Early Days of Interactive Information Visualization at the University of Maryland.

The thrill of scientific discovery, the excitement of engineering development, and the fresh thinking of design explorations were invigorating as we participated in the birth of a new discipline: Information Visualization. This discipline, based on graphical user interfaces with pointing devices, became possible as software matured, hardware sped up, and screen resolution improved. Driven by the concepts of direct manipulation and dynamic queries, we made interactive interfaces that empowered users and opened up new possibilities for the next generation of designers. We worked with professionals who had real problems and tested real users to get their feedback. Some projects failed and some papers never got published, but many of the new ideas found their way into widely used commercial products. Our great satisfaction is that our students have spread the community spirit of the Human-Computer Interaction Laboratory as they continue to make further contributions.

Twin-Win Model: A human-centered approach to research success.

A 70-year-old simmering debate has erupted into vigorous battles over the most effective ways to conduct research. Well-established beliefs are being forcefully challenged by advocates of new research models. While there can be no final resolution to this battle, this paper offers the Twin-Win Model to guide teams of researchers, academic leaders, business managers, and government funding policymakers. The Twin-Win Model favors a problem-oriented approach to research, which encourages formation of teams to pursue the dual goals of breakthrough theories in published papers and validated solutions that are ready for widespread dissemination. The raised expectations of simultaneously pursuing foundational discoveries and powerful innovations are a step beyond traditional approaches that advocate basic research first. Evidence from citation analysis and researcher interviews suggests that simultaneous pursuit of both goals raises the chance of twin-win success.

Apply or Die: On the Role and Assessment of Application Papers in Visualization.

Application-oriented papers provide an important way to invigorate and cross-pollinate the visualization field, but the exact criteria for judging an application paper's merit remain an open question. This article builds on a panel at the 2016 IEEE Visualization Conference entitled "Application Papers: What Are They, and How Should They Be Evaluated?" that sought to gain a better understanding of prevalent views in the visualization community. This article surveys current trends that favor application papers, reviews the benefits and contributions of this paper type, and discusses their assessment in the review process. It concludes with recommendations to ensure that the visualization community is more inclusive to application papers.

Coping with Volume and Variety in Temporal Event Sequences: Strategies for Sharpening Analytic Focus.

The growing volume and variety of data presents both opportunities and challenges for visual analytics. Addressing these challenges is needed for big data to provide valuable insights and novel solutions for business, security, social media, and healthcare. In the case of temporal event sequence analytics it is the number of events in the data and variety of temporal sequence patterns that challenges users of visual analytic tools. This paper describes 15 strategies for sharpening analytic focus that analysts can use to reduce the data volume and pattern variety. Four groups of strategies are proposed: (1) extraction strategies, (2) temporal folding, (3) pattern simplification strategies, and (4) iterative strategies. For each strategy, we provide examples of the use and impact of this strategy on volume and/or variety. Examples are selected from 20 case studies gathered from either our own work, the literature, or based on email interviews with individuals who conducted the analyses and developers who observed analysts using the tools. Finally, we discuss how these strategies might be combined and report on the feedback from 10 senior event sequence analysts.

Novel user interface design for medication reconciliation: an evaluation of Twinlist.

OBJECTIVE: The primary objective was to evaluate time, number of interface actions, and accuracy on medication reconciliation tasks using a novel user interface (Twinlist, which lays out the medications in five columns based on similarity and uses animation to introduce the grouping - www.cs.umd.edu/hcil/sharp/twinlist) compared to a Control interface (where medications are presented side by side in two columns). A secondary objective was to assess participant agreement with statements regarding clarity and utility and to elicit comparisons. MATERIAL AND METHODS: A 1 × 2 within-subjects experimental design was used with interface (Twinlist or Control) as an independent variable; time, number of clicks, scrolls, and errors were used as dependent variables. Participants were practicing medical providers with experience performing medication reconciliation but no experience with Twinlist. They reconciled two cases in each interface (in a counterbalanced order), then provided feedback on the design of the interface. RESULTS: Twenty medical providers participated in the study for a total of 80 trials. The trials using Twinlist were statistically significantly faster (18%), with fewer clicks (40%) and scrolls (60%). Serious errors were noted 12 and 31 times in Twinlist and Control trials, respectively. DISCUSSION: Trials using Twinlist were faster and more accurate. Subjectively, participants rated Twinlist more favorably than Control. They valued the novel layout of the drugs, but indicated that the included animation would be valuable for novices, but not necessarily for advanced users. Additional feedback from participants provides guidance for further development and clinical implementations. CONCLUSIONS: Cognitive support of medication reconciliation through interface design can significantly improve performance and safety.

A task taxonomy for network evolution analysis.

Visualization has proven to be a useful tool for understanding network structures. Yet the dynamic nature of social media networks requires powerful visualization techniques that go beyond static network diagrams. To provide strong temporal network visualization tools, designers need to understand what tasks the users have to accomplish. This paper describes a taxonomy of temporal network visualization tasks. We identify the 1) entities, 2) properties, and 3) temporal features, which were extracted by surveying 53 existing temporal network visualization systems. By building and examining the task taxonomy, we report which tasks are well covered by existing systems and make suggestions for designing future visualization tools. The feedback from 12 network analysts helped refine the taxonomy.

Reducing wrong patient selection errors: exploring the design space of user interface techniques.

Wrong patient selection errors are a major issue for patient safety; from ordering medication to performing surgery, the stakes are high. Widespread adoption of Electronic Health Record (EHR) and Computerized Provider Order Entry (CPOE) systems makes patient selection using a computer screen a frequent task for clinicians. Careful design of the user interface can help mitigate the problem by helping providers recall their patients' identities, accurately select their names, and spot errors before orders are submitted. We propose a catalog of twenty seven distinct user interface techniques, organized according to a task analysis. An associated video demonstrates eighteen of those techniques. EHR designers who consider a wider range of human-computer interaction techniques could reduce selection errors, but verification of efficacy is still needed.

Temporal event sequence simplification.

Electronic Health Records (EHRs) have emerged as a cost-effective data source for conducting medical research. The difficulty in using EHRs for research purposes, however, is that both patient selection and record analysis must be conducted across very large, and typically very noisy datasets. Our previous work introduced EventFlow, a visualization tool that transforms an entire dataset of temporal event records into an aggregated display, allowing researchers to analyze population-level patterns and trends. As datasets become larger and more varied, however, it becomes increasingly difficult to provide a succinct, summarizing display. This paper presents a series of user-driven data simplifications that allow researchers to pare event records down to their core elements. Furthermore, we present a novel metric for measuring visual complexity, and a language for codifying disjoint strategies into an overarching simplification framework. These simplifications were used by real-world researchers to gain new and valuable insights from initially overwhelming datasets.

Visualizing change over time using dynamic hierarchies: TreeVersity2 and the StemView.

To analyze data such as the US Federal Budget or characteristics of the student population of a University it is common to look for changes over time. This task can be made easier and more fruitful if the analysis is performed by grouping by attributes, such as by Agencies, Bureaus and Accounts for the Budget, or Ethnicity, Gender and Major in a University. We present TreeVersity2, a web based interactive data visualization tool that allows users to analyze change in datasets by creating dynamic hierarchies based on the data attributes. TreeVersity2 introduces a novel space filling visualization (StemView) to represent change in trees at multiple levels--not just at the leaf level. With this visualization users can explore absolute and relative changes, created and removed nodes, and each node's actual values, while maintaining the context of the tree. In addition, TreeVersity2 provides overviews of change over the entire time period, and a reporting tool that lists outliers in textual form, which helps users identify the major changes in the data without having to manually setup filters. We validated TreeVersity2 with 12 case studies with organizations as diverse as the National Cancer Institute, Federal Drug Administration, Department of Transportation, Office of the Bursar of the University of Maryland, or eBay. Our case studies demonstrated that TreeVersity2 is flexible enough to be used in different domains and provide useful insights for the data owners. A TreeVersity2 demo can be found at https://treeversity.cattlab.umd.edu.

An information-centric framework for designing patient-centered medical decision aids and risk communication.

Risk communication is a major challenge in productive patient-physician communication. Patient decision making responsibilities come with an implicit assumption that patients are sufficiently educated and confident in their abilities to make decisions about their care based on evidence based treatment recommendations. Attempts to improve health literacy in patients by way of graphical decision aids have met with success. Such decision aids typically have been designed for a general population and evaluated based on whether or not users of the decision aid can accurately report the data points in isolation. To classify decision aids, we present an information-centric framework for assessing the content delivered to patients. We provide examples of our framework from a literature survey and suggest ways improvements can be made by considering all dimensions of our framework.

Twinlist: novel user interface designs for medication reconciliation.

Medication reconciliation is an important and complex task for which careful user interface design has the potential to help reduce errors and improve quality of care. In this paper we focus on the hospital discharge scenario and first describe a novel interface called Twinlist. Twinlist illustrates the novel use of spatial layout combined with multi-step animation, to help medical providers see what is different and what is similar between the lists (e.g., intake list and hospital list), and rapidly choose the drugs they want to include in the reconciled list. We then describe a series of variant designs and discuss their comparative advantages and disadvantages. Finally we report on a pilot study that suggests that animation might help users learn new spatial layouts such as the one used in Twinlist.

Querying Event Sequences by Exact Match or Similarity Search: Design and Empirical Evaluation.

Specifying event sequence queries is challenging even for skilled computer professionals familiar with SQL. Most graphical user interfaces for database search use an exact match approach, which is often effective, but near misses may also be of interest. We describe a new similarity search interface, in which users specify a query by simply placing events on a blank timeline and retrieve a similarity-ranked list of results. Behind this user interface is a new similarity measure for event sequences which the users can customize by four decision criteria, enabling them to adjust the impact of missing, extra, or swapped events or the impact of time shifts. We describe a use case with Electronic Health Records based on our ongoing collaboration with hospital physicians. A controlled experiment with 18 participants compared exact match and similarity search interfaces. We report on the advantages and disadvantages of each interface and suggest a hybrid interface combining the best of both.

Medication Reconciliation: Work Domain Ontology, prototype development, and a predictive model.

Medication errors can result from administration inaccuracies at any point of care and are a major cause for concern. To develop a successful Medication Reconciliation (MR) tool, we believe it necessary to build a Work Domain Ontology (WDO) for the MR process. A WDO defines the explicit, abstract, implementation-independent description of the task by separating the task from work context, application technology, and cognitive architecture. We developed a prototype based upon the WDO and designed to adhere to standard principles of interface design. The prototype was compared to Legacy Health System's and Pre-Admission Medication List Builder MR tools via a Keystroke-Level Model analysis for three MR tasks. The analysis found the prototype requires the fewest mental operations, completes tasks in the fewest steps, and completes tasks in the least amount of time. Accordingly, we believe that developing a MR tool, based upon the WDO and user interface guidelines, improves user efficiency and reduces cognitive load.

Reducing missed laboratory results: defining temporal responsibility, generating user interfaces for test process tracking, and retrospective analyses to identify problems.

Researchers have conducted numerous case studies reporting the details on how laboratory test results of patients were missed by the ordering medical providers. Given the importance of timely test results in an outpatient setting, there is limited discussion of electronic versions of test result management tools to help clinicians and medical staff with this complex process. This paper presents three ideas to reduce missed results with a system that facilitates tracking laboratory tests from order to completion as well as during follow-up: (1) define a workflow management model that clarifies responsible agents and associated time frame, (2) generate a user interface for tracking that could eventually be integrated into current electronic health record (EHR) systems, (3) help identify common problems in past orders through retrospective analyses.

Extracting insights from electronic health records: case studies, a visual analytics process model, and design recommendations.

Current electronic health record (EHR) systems facilitate the storage, retrieval, persistence, and sharing of patient data. However, the way physicians interact with EHRs has not changed much. More specifically, support for temporal analysis of a large number of EHRs has been lacking. A number of information visualization techniques have been proposed to alleviate this problem. Unfortunately, due to their limited application to a single case study, the results are often difficult to generalize across medical scenarios. We present the usage data of Lifelines2 (Wang et al. 2008), our information visualization system, and user comments, both collected over eight different medical case studies. We generalize our experience into a visual analytics process model for multiple EHRs. Based on our analysis, we make seven design recommendations to information visualization tools to explore EHR systems.