Perceptually Uniform Construction of Illustrative Textures.

Illustrative textures, such as stippling or hatching, were predominantly used as an alternative to conventional Phong rendering. Recently, the potential of encoding information on surfaces or maps using different densities has also been recognized. This has the significant advantage that additional color can be used as another visual channel and the illustrative textures can then be overlaid. Effectively, it is thus possible to display multiple information, such as two different scalar fields on surfaces simultaneously. In previous work, these textures were manually generated and the choice of density was unempirically determined. Here, we first want to determine and understand the perceptual space of illustrative textures. We chose a succession of simplices with increasing dimensions as primitives for our textures: Dots, lines, and triangles. Thus, we explore the texture types of stippling, hatching, and triangles. We create a range of textures by sampling the density space uniformly. Then, we conduct three perceptual studies in which the participants performed pairwise comparisons for each texture type. We use multidimensional scaling (MDS) to analyze the perceptual spaces per category. The perception of stippling and triangles seems relatively similar. Both are adequately described by a 1D manifold in 2D space. The perceptual space of hatching consists of two main clusters: Crosshatched textures, and textures with only one hatching direction. However, the perception of hatching textures with only one hatching direction is similar to the perception of stippling and triangles. Based on our findings, we construct perceptually uniform illustrative textures. Afterwards, we provide concrete application examples for the constructed textures.

A Survey of Medical Visualization through the Lens of Metaphors.

We provide an overview of metaphors that were used in medical visualization and related user interfaces. Metaphors are employed to translate concepts from a source domain to a target domain. The survey is grounded in a discussion of metaphor-based design involving the identification and reflection of candidate metaphors. We consider metaphors that have a source domain in one branch of medicine, e.g., the virtual mirror that solves problems in orthopedics and laparoscopy with a mirror that resembles the dentist's mirror. Other metaphors employ the physical world as the source domain, such as crepuscular rays that inspire a solution for access planning in tumor therapy. Aviation is another source of inspiration, leading to metaphors, such as surgical cockpits, surgical control towers, and surgery navigation according to an instrument flight. This paper should raise awareness for metaphors and their potential to focus the design of computer-assisted systems on useful features and a positive user experience. Limitations and potential drawbacks of a metaphor-based user interface design for medical applications are also considered.

Predicting the HER2 status in oesophageal cancer from tissue microarrays using convolutional neural networks.

BACKGROUND: Fast and accurate diagnostics are key for personalised medicine. Particularly in cancer, precise diagnosis is a prerequisite for targeted therapies, which can prolong lives. In this work, we focus on the automatic identification of gastroesophageal adenocarcinoma (GEA) patients that qualify for a personalised therapy targeting epidermal growth factor receptor 2 (HER2). We present a deep-learning method for scoring microscopy images of GEA for the presence of HER2 overexpression. METHODS: Our method is based on convolutional neural networks (CNNs) trained on a rich dataset of 1602 patient samples and tested on an independent set of 307 patient samples. We additionally verified the CNN's generalisation capabilities with an independent dataset with 653 samples from a separate clinical centre. We incorporated an attention mechanism in the network architecture to identify the tissue regions, which are important for the prediction outcome. Our solution allows for direct automated detection of HER2 in immunohistochemistry-stained tissue slides without the need for manual assessment and additional costly in situ hybridisation (ISH) tests. RESULTS: We show accuracy of 0.94, precision of 0.97, and recall of 0.95. Importantly, our approach offers accurate predictions in cases that pathologists cannot resolve and that require additional ISH testing. We confirmed our findings in an independent dataset collected in a different clinical centre. The attention-based CNN exploits morphological information in microscopy images and is superior to a predictive model based on the staining intensity only. CONCLUSIONS: We demonstrate that our approach not only automates an important diagnostic process for GEA patients but also paves the way for the discovery of new morphological features that were previously unknown for GEA pathology.

GUCCI - Guided Cardiac Cohort Investigation of Blood Flow Data.

We present the framework GUCCI (Guided Cardiac Cohort Investigation), which provides a guided visual analytics workflow to analyze cohort-based measured blood flow data in the aorta. In the past, many specialized techniques have been developed for the visual exploration of such data sets for a better understanding of the influence of morphological and hemodynamic conditions on cardiovascular diseases. However, there is a lack of dedicated techniques that allow visual comparison of multiple data sets and defined cohorts, which is essential to characterize pathologies. GUCCI offers visual analytics techniques and novel visualization methods to guide the user through the comparison of predefined cohorts, such as healthy volunteers and patients with a pathologically altered aorta. The combination of overview and glyph-based depictions together with statistical cohort-specific information allows investigating differences and similarities of the time-dependent data. Our framework was evaluated in a qualitative user study with three radiologists specialized in cardiac imaging and two experts in medical blood flow visualization. They were able to discover cohort-specific characteristics, which supports the derivation of standard values as well as the assessment of pathology-related severity and the need for treatment.

Visual Assistance in Development and Validation of Bayesian Networks for Clinical Decision Support.

The development and validation of Clinical Decision Support Models (CDSM) based on Bayesian networks (BN) is commonly done in a collaborative work between medical researchers providing the domain expertise and computer scientists developing the decision support model. Although modern tools provide facilities for data-driven model generation, domain experts are required to validate the accuracy of the learned model and to provide expert knowledge for fine-tuning it while computer scientists are needed to integrate this knowledge in the learned model (hybrid modeling approach). This generally time-expensive procedure hampers CDSM generation and updating. To address this problem, we developed a novel interactive visual approach allowing medical researchers with less knowledge in CDSM to develop and validate BNs based on domain specific data mainly independently and thus, diminishing the need for an additional computer scientist. In this context, we abstracted and simplified the common workflow in BN development as well as adjusted the workflow to medical experts' needs. We demonstrate our visual approach with data of endometrial cancer patients and evaluated it with six medical researchers who are domain experts in the gynecological field.

GRay: Ray Casting for Visualization and Interactive Data Exploration of Gaussian Mixture Models.

The Gaussian mixture model (GMM) describes the distribution of random variables from several different populations. GMMs have widespread applications in probability theory, statistics, machine learning for unsupervised cluster analysis and topic modeling, as well as in deep learning pipelines. So far, few efforts have been made to explore the underlying point distribution in combination with the GMMs, in particular when the data becomes high-dimensional and when the GMMs are composed of many Gaussians. We present an analysis tool comprising various GPU-based visualization techniques to explore such complex GMMs. To facilitate the exploration of high-dimensional data, we provide a novel navigation system to analyze the underlying data. Instead of projecting the data to 2D, we utilize interactive 3D views to better support users in understanding the spatial arrangements of the Gaussian distributions. The interactive system is composed of two parts: (1) raycasting-based views that visualize cluster memberships, spatial arrangements, and support the discovery of new modes. (2) overview visualizations that enable the comparison of Gaussians with each other, as well as small multiples of different choices of basis vectors. Users are supported in their exploration with customization tools and smooth camera navigations. Our tool was developed and assessed by five domain experts, and its usefulness was evaluated with 23 participants. To demonstrate the effectiveness, we identify interesting features in several data sets.

Interactive visual exploration of surgical process data.

PURPOSE: Integrated operating rooms provide rich sources of temporal information about surgical procedures, which has led to the emergence of surgical data science. However, little emphasis has been put on interactive visualization of such temporal datasets to gain further insights. Our goal is to put heterogeneous data sequences in relation to better understand the workflows of individual procedures as well as selected subsets, e.g., with respect to different surgical phase distributions and surgical instrument usage patterns. METHODS: We developed a reusable web-based application design to analyze data derived from surgical procedure recordings. It consists of aggregated, synchronized visualizations for the original temporal data as well as for derived information, and includes tailored interaction techniques for selection and filtering. To enable reproducibility, we evaluated it across four types of surgeries from two openly available datasets (HeiCo and Cholec80). User evaluation has been conducted with twelve students and practitioners with surgical and technical background. RESULTS: The evaluation showed that the application has the complexity of an expert tool (System Usability Score of 57.73) but allowed the participants to solve various analysis tasks correctly (78.8% on average) and to come up with novel hypotheses regarding the data. CONCLUSION: The novel application supports postoperative expert-driven analysis, improving the understanding of surgical workflows and the underlying datasets. It facilitates analysis across multiple synchronized views representing information from different data sources and, thereby, advances the field of surgical data science.

Classification of Blood Flow Patterns in Cerebral Aneurysms.

We present a Cerebral Aneurysm Vortex Classification (CAVOCLA) that allows to classify blood flow in cerebral aneurysms. Medical studies assume a strong relation between the progression and rupture of aneurysms and flow patterns. To understand how flow patterns impact the vessel morphology, they are manually classified according to predefined classes. However, manual classifications are time-consuming and exhibit a high inter-observer variability. In contrast, our approach is more objective and faster than manual methods. The classification of integral lines, representing steady or unsteady blood flow, is based on a mapping of the aneurysm surface to a hemisphere by calculating polar-based coordinates. The lines are clustered and for each cluster a representative is calculated. Then, the polar-based coordinates are transformed to the representative as basis for the classification. Classes are based on the flow complexity. The classification results are presented by a detail-on-demand approach using a visual transition from the representative over an enclosing surface to the associated lines. Based on seven representative datasets, we conduct an informal interview with five domain experts to evaluate the system. They confirmed that CAVOCLA allows for a robust classification of intra-aneurysmal flow patterns. The detail-on-demand visualization enables an efficient exploration and interpretation of flow patterns.

Visual Analysis of Aneurysm Data using Statistical Graphics.

This paper presents a framework to explore multi-field data of aneurysms occurring at intracranial and cardiac arteries by using statistical graphics. The rupture of an aneurysm is often a fatal scenario, whereas during treatment serious complications for the patient can occur. Whether an aneurysm ruptures or whether a treatment is successful depends on the interaction of different morphological such as wall deformation and thickness, and hemodynamic attributes like wall shear stress and pressure. Therefore, medical researchers are very interested in better understanding these relationships. However, the required analysis is a time-consuming process, where suspicious wall regions are difficult to detect due to the time-dependent behavior of the data. Our proposed visualization framework enables medical researchers to efficiently assess aneurysm risk and treatment options. This comprises a powerful set of views including 2D and 3D depictions of the aneurysm morphology as well as statistical plots of different scalar fields. Brushing and linking aids the user to identify interesting wall regions and to understand the influence of different attributes on the aneurysm's state. Moreover, a visual comparison of pre- and post-treatment as well as different treatment options is provided. Our analysis techniques are designed in collaboration with domain experts, e.g., physicians, and we provide details about the evaluation.

Management of Cerebral Aneurysm Descriptors based on an Automatic Ostium Extraction.

We present a framework to manage cerebral aneurysms. Rupture risk evaluation is based on manually extracted descriptors, which is time-consuming. Thus, we provide an automatic solution by considering several questions: How can expert knowledge be integrated? How should meta data be defined? Which interaction techniques are needed for data exploration.

Combined Visualization of Vessel Deformation and Hemodynamics in Cerebral Aneurysms.

We present the first visualization tool that combines patient-specific hemodynamics with information about the vessel wall deformation and wall thickness in cerebral aneurysms. Such aneurysms bear the risk of rupture, whereas their treatment also carries considerable risks for the patient. For the patient-specific rupture risk evaluation and treatment analysis, both morphological and hemodynamic data have to be investigated. Medical researchers emphasize the importance of analyzing correlations between wall properties such as the wall deformation and thickness, and hemodynamic attributes like the Wall Shear Stress and near-wall flow. Our method uses a linked 2.5D and 3D depiction of the aneurysm together with blood flow information that enables the simultaneous exploration of wall characteristics and hemodynamic attributes during the cardiac cycle. We thus offer medical researchers an effective visual exploration tool for aneurysm treatment risk assessment. The 2.5D view serves as an overview that comprises a projection of the vessel surface to a 2D map, providing an occlusion-free surface visualization combined with a glyph-based depiction of the local wall thickness. The 3D view represents the focus upon which the data exploration takes place. To support the time-dependent parameter exploration and expert collaboration, a camera path is calculated automatically, where the user can place landmarks for further exploration of the properties. We developed a GPU-based implementation of our visualizations with a flexible interactive data exploration mechanism. We designed our techniques in collaboration with domain experts, and provide details about the evaluation.