Subjective perception of visual field defects using random noise-moving images in patients with glaucoma: A comparison of computer graphics and analog noises.

PURPOSE: Random noise-moving images (noises) can make glaucoma patients with no subjective symptoms aware of visual field abnormalities. To explore this concept, we developed a noise using computer graphics (CG) and investigated the difference in the subjective perception of visual field abnormalities between CG and conventional analog noises. METHODS: We enrolled individuals with glaucoma (205 eyes), preperimetric glaucoma (PPG; 19 eyes), and normal eyes (35 eyes). For a CG noise, a series of still images was made by randomly selecting five monochromatic tones on 2-mm square dots, and these images were drawn at 60 frames per second (fps) to create a noise-moving image. The participants were asked to describe their perceived shadows on a paper. The results were categorized as follows based on the pattern deviation probability map of the Humphrey field analyzer (HFA): "agreement," "partial agreement," "disagreement," and "no response." The glaucoma stage was classified into four stages, from M1 to M4, based on the HFA's mean deviation. RESULT: The detection rates (agreement and partial agreement) were 80.5% and 65.4% for the CG and analog noises, respectively, with CG noise showing a significantly higher detection rate in all glaucoma eyes (P < 0.001). The detection rates tended to increase as the glaucoma stage progressed, and in Stage M3, these were 93.9% and 78.8% for the CG and analog noises, respectively. The PPG eyes did not exhibit subjective abnormalities for both noises. The specificity values were 97.1% and 100% for the CG and analog noises, respectively. CONCLUSION: The CG noise is more effective than the analog noise in evaluating the subjective perception of visual field abnormalities in patients with glaucoma.

Sparse Spiking Neural-Like Membrane Systems on Graphics Processing Units.

The parallel simulation of Spiking Neural P systems is mainly based on a matrix representation, where the graph inherent to the neural model is encoded in an adjacency matrix. The simulation algorithm is based on a matrix-vector multiplication, which is an operation efficiently implemented on parallel devices. However, when the graph of a Spiking Neural P system is not fully connected, the adjacency matrix is sparse and hence, lots of computing resources are wasted in both time and memory domains. For this reason, two compression methods for the matrix representation were proposed in a previous work, but they were not implemented nor parallelized on a simulator. In this paper, they are implemented and parallelized on GPUs as part of a new Spiking Neural P system with delays simulator. Extensive experiments are conducted on high-end GPUs (RTX2080 and A100 80GB), and it is concluded that they outperform other solutions based on state-of-the-art GPU libraries when simulating Spiking Neural P systems.

PROTAX-GPU: a scalable probabilistic taxonomic classification system for DNA barcodes.

DNA-based identification is vital for classifying biological specimens, yet methods to quantify the uncertainty of sequence-based taxonomic assignments are scarce. Challenges arise from noisy reference databases, including mislabelled entries and missing taxa. PROTAX addresses these issues with a probabilistic approach to taxonomic classification, advancing on methods that rely solely on sequence similarity. It provides calibrated probabilistic assignments to a partially populated taxonomic hierarchy, accounting for taxa that lack references and incorrect taxonomic annotation. While effective on smaller scales, global application of PROTAX necessitates substantially larger reference libraries, a goal previously hindered by computational barriers. We introduce PROTAX-GPU, a scalable algorithm capable of leveraging the global Barcode of Life Data System (>14 million specimens) as a reference database. Using graphics processing units (GPU) to accelerate similarity and nearest-neighbour operations and the JAX library for Python integration, we achieve over a 1000 × speedup compared with the central processing unit (CPU)-based implementation without compromising PROTAX's key benefits. PROTAX-GPU marks a significant stride towards real-time DNA barcoding, enabling quicker and more efficient species identification in environmental assessments. This capability opens up new avenues for real-time monitoring and analysis of biodiversity, advancing our ability to understand and respond to ecological dynamics. This article is part of the theme issue 'Towards a toolkit for global insect biodiversity monitoring'.

OmicNavigator: open-source software for the exploration, visualization, and archival of omic studies.

BACKGROUND: The results of high-throughput biology ('omic') experiments provide insight into biological mechanisms but can be challenging to explore, archive and share. The scale of these challenges continues to grow as omic research volume expands and multiple analytical technologies, bioinformatic pipelines, and visualization preferences have emerged. Multiple software applications exist that support omic study exploration and/or archival. However, an opportunity remains for open-source software that can archive and present the results of omic analyses with broad accommodation of study-specific analytical approaches and visualizations with useful exploration features. RESULTS: We present OmicNavigator, an R package for the archival, visualization and interactive exploration of omic studies. OmicNavigator enables bioinformaticians to create web applications that interactively display their custom visualizations and analysis results linked with app-derived analytical tools, graphics, and tables. Studies created with OmicNavigator can be viewed within an interactive R session or hosted on a server for shared access. CONCLUSIONS: OmicNavigator can be found at https://github.com/abbvie-external/OmicNavigator.

Real-time soft body dissection simulation with parallelized graph-based shape matching on GPU.

BACKGROUND AND OBJECTIVE: Interactive soft tissue dissection has been a fundamental procedure in virtual surgery systems. Existing cutting algorithms involve complex topology changes of simulation meshes, which can increase simulation overhead and produce visual artifacts. In this paper, we proposed a novel graph-based shape-matching method that allows for real-time, flexible, progressive, and discontinuous cuts on soft tissue. METHODS: We employed shape-matching constraints within the position-based dynamics (PBD) framework, a widely adopted approach for real-time simulation applications. The soft tissue was effectively modeled using overlapping clusters, each governed by shape-matching constraints. The dissection process was bifurcated into two distinct stages. In the first stage, the surgical scalpel presses the surface of the soft tissue. The soft tissue is cut apart when the surface pressure exceeds a threshold, entering the second stage. To address the discrepancy between the visual mesh and the simulation model during cluster separation, we developed an Aggregate Finding Connected Components (AFCC) algorithm, optimized for GPU computation and integrated with a background grid. This approach also avoids ghost forces and fragmentation artifacts. To control the increase in the number of clusters, we also propose a merging strategy that can run in parallel. RESULTS: Our simulation outcomes demonstrated that the AFCC dissection algorithm effectively manages cluster separation and expansion with robustness. There were no ghost forces between the cutting surface and unrealistic fragments. Our simulation capability extended to supporting intricate and discontinuous cutting routes. Our dissection simulation maintained real-time performance even with over 100,000 particles constituting the soft tissue. CONCLUSIONS: Our real-time and robust surgical dissection simulation technique enables the performance of complex cuts in various surgical scenarios, demonstrating its potential in virtual surgery applications.

IOeRT conventional and FLASH treatment planning system implementation exploiting fast GPU Monte Carlo: The case of breast cancer.

Partial breast irradiation for the treatment of early-stage breast cancer patients can be performed by means of Intra Operative electron Radiation Therapy (IOeRT). One of the main limitations of this technique is the absence of a treatment planning system (TPS) that could greatly help in ensuring a proper coverage of the target volume during irradiation. An IOeRT TPS has been developed using a fast Monte Carlo (MC) and an ultrasound imaging system to provide the best irradiation strategy (electron beam energy, applicator position and bevel angle) and to facilitate the optimisation of dose prescription and delivery to the target volume while maximising the organs at risk sparing. The study has been performed in silico, exploiting MC simulations of a breast cancer treatment. Ultrasound-based input has been used to compute the absorbed dose maps in different irradiation strategies and a quantitative comparison between the different options was carried out using Dose Volume Histograms. The system was capable of exploring different beam energies and applicator positions in few minutes, identifying the best strategy with an overall computation time that was found to be completely compatible with clinical implementation. The systematic uncertainty related to tissue deformation during treatment delivery with respect to imaging acquisition was taken into account. The potential and feasibility of a GPU based full MC TPS implementation of IOeRT breast cancer treatments has been demonstrated in-silico. This long awaited tool will greatly improve the treatment safety and efficacy, overcoming the limits identified within the clinical trials carried out so far.

A novel interactive deep cascade spectral graph convolutional network with multi-relational graphs for disease prediction.

Graph neural networks (GNNs) have recently grown in popularity for disease prediction. Existing GNN-based methods primarily build the graph topological structure around a single modality and combine it with other modalities to acquire feature representations of acquisitions. The complicated relationship in each modality, however, may not be well highlighted due to its specificity. Further, relatively shallow networks restrict adequate extraction of high-level features, affecting disease prediction performance. Accordingly, this paper develops a new interactive deep cascade spectral graph convolutional network with multi-relational graphs (IDCGN) for disease prediction tasks. Its crucial points lie in constructing multiple relational graphs and dual cascade spectral graph convolution branches with interaction (DCSGBI). Specifically, the former designs a pairwise imaging-based edge generator and a pairwise non-imaging-based edge generator from different modalities by devising two learnable networks, which adaptively capture graph structures and provide various views of the same acquisition to aid in disease diagnosis. Again, DCSGBI is established to enrich high-level semantic information and low-level details of disease data. It devises a cascade spectral graph convolution operator for each branch and incorporates the interaction strategy between different branches into the network, successfully forming a deep model and capturing complementary information from diverse branches. In this manner, more favorable and sufficient features are learned for a reliable diagnosis. Experiments on several disease datasets reveal that IDCGN exceeds state-of-the-art models and achieves promising results.

GPU-Based Parallel Processing Techniques for Enhanced Brain Magnetic Resonance Imaging Analysis: A Review of Recent Advances.

The approach of using more than one processor to compute in order to overcome the complexity of different medical imaging methods that make up an overall job is known as GPU (graphic processing unit)-based parallel processing. It is extremely important for several medical imaging techniques such as image classification, object detection, image segmentation, registration, and content-based image retrieval, since the GPU-based parallel processing approach allows for time-efficient computation by a software, allowing multiple computations to be completed at once. On the other hand, a non-invasive imaging technology that may depict the shape of an anatomy and the biological advancements of the human body is known as magnetic resonance imaging (MRI). Implementing GPU-based parallel processing approaches in brain MRI analysis with medical imaging techniques might be helpful in achieving immediate and timely image capture. Therefore, this extended review (the extension of the IWBBIO2023 conference paper) offers a thorough overview of the literature with an emphasis on the expanding use of GPU-based parallel processing methods for the medical analysis of brain MRIs with the imaging techniques mentioned above, given the need for quicker computation to acquire early and real-time feedback in medicine. Between 2019 and 2023, we examined the articles in the literature matrix that include the tasks, techniques, MRI sequences, and processing results. As a result, the methods discussed in this review demonstrate the advancements achieved until now in minimizing computing runtime as well as the obstacles and problems still to be solved in the future.

MusiKeys: Exploring Haptic-to-Auditory Sensory Substitution to Improve Mid-Air Text-Entry.

Physical QWERTY keyboards are the current standard for performing precision text-entry with extended reality devices. Ideally, there would exist a comparable, self-contained solution that works anywhere, without requiring external keyboards. Unfortunately, when physical keyboards are recreated virtually, we currently lose critical haptic feedback information from the sense of touch, which impedes typing. In this paper, we introduce the MusiKeys Technique, which uses auditory feedback in virtual reality to communicate missing haptic feedback information typists normally receive when using a physical keyboard. To examine this concept, we conducted a user study with 24 participants which encompassed four mid-air virtual keyboards augmented with increasing amounts of feedback information, along with a fifth physical keyboard for reference. Results suggest that providing clicking feedback on key-press and key-release improves typing performance compared to not providing auditory feedback, which is consistent with the literature. We also found that audio can serve as a substitute for information contained in haptic feedback, in that users can accurately perceive the presented information. However, under our specific study conditions, this awareness of the feedback information did not yield significant differences in typing performance. Our results suggest this kind of feedback replacement can be perceived by users but needs more research to tune and improve the specific techniques.

Swift-Eye: Towards Anti-blink Pupil Tracking for Precise and Robust High-Frequency Near-Eye Movement Analysis with Event Cameras.

Eye tracking has shown great promise in many scientific fields and daily applications, ranging from the early detection of mental health disorders to foveated rendering in virtual reality (VR). These applications all call for a robust system for high-frequency near-eye movement sensing and analysis in high precision, which cannot be guaranteed by the existing eye tracking solutions with CCD/CMOS cameras. To bridge the gap, in this paper, we propose Swift-Eye, an offline precise and robust pupil estimation and tracking framework to support high-frequency near-eye movement analysis, especially when the pupil region is partially occluded. Swift-Eye is built upon the emerging event cameras to capture the high-speed movement of eyes in high temporal resolution. Then, a series of bespoke components are designed to generate high-quality near-eye movement video at a high frame rate over kilohertz and deal with the occlusion over the pupil caused by involuntary eye blinks. According to our extensive evaluations on EV-Eye, a large-scale public dataset for eye tracking using event cameras, Swift-Eye shows high robustness against significant occlusion. It can improve the IoU and F1-score of the pupil estimation by 20% and 12.5% respectively, compared with the second-best competing approach, when over 80% of the pupil region is occluded by the eyelid. Lastly, it provides continuous and smooth traces of pupils in extremely high temporal resolution and can support high-frequency eye movement analysis and a number of potential applications, such as mental health diagnosis, behaviour-brain association, etc. The implementation details and source codes can be found at https://github.com/ztysdu/Swift-Eye.

Comparing Synchronous and Asynchronous Task Delivery in Mixed Reality Environments.

Asynchronous digital communication is a widely applied and well-known form of information exchange. Most pieces of technology make use of some variation of asynchronous communication systems, be it messaging or email applications. This allows recipients to process digital messages immediately (synchronous) or whenever they have time (asynchronous), meaning that purely digital interruptions can be mitigated easily. Mixed Reality systems have the potential to not only handle digital interruptions but also interruptions in physical space, e.g., caused by co-workers in workspaces or learning environments. However, the benefits of such systems previously remained untested in the context of Mixed Reality. We conducted a user study ($\mathrm{N}=26$) to investigate the impact that the timing of task delivery has on the participants' performance, workflow, and emotional state. Participants had to perform several cognitively demanding tasks in a Mixed Reality workspace. Inside the virtual workspace, we simulated in-person task delivery either during tasks (i.e., interrupting the participant) or between tasks (i.e., delaying the interruption). Our results show that delaying interruptions has a significant impact on subjective metrics like the perceived performance and workload.

Exploring the Influence of Virtual Avatar Heads in Mixed Reality on Social Presence, Performance and User Experience in Collaborative Tasks.

In Mixed Reality (MR), users' heads are largely (if not completely) occluded by the MR Head-Mounted Display (HMD) they are wearing. As a consequence, one cannot see their facial expressions and other communication cues when interacting locally. In this paper, we investigate how displaying virtual avatars' heads on-top of the (HMD-occluded) heads of participants in a Video See-Through (VST) Mixed Reality local collaborative task could improve their collaboration as well as social presence. We hypothesized that virtual heads would convey more communicative cues (such as eye direction or facial expressions) hidden by the MR HMDs and lead to better collaboration and social presence. To do so, we conducted a between-subject study ($\mathrm{n}=88$) with two independent variables: the type of avatar (CartoonAvatar/RealisticAvatar/NoAvatar) and the level of facial expressions provided (HighExpr/LowExpr). The experiment involved two dyadic communication tasks: (i) the "20-question" game where one participant asks questions to guess a hidden word known by the other participant and (ii) a urban planning problem where participants have to solve a puzzle by collaborating. Each pair of participants performed both tasks using a specific type of avatar and facial animation. Our results indicate that while adding an avatar's head does not necessarily improve social presence, the amount of facial expressions provided through the social interaction does have an impact. Moreover, participants rated their performance higher when observing a realistic avatar but rated the cartoon avatars as less uncanny. Taken together, our results contribute to a better understanding of the role of partial avatars in local MR collaboration and pave the way for further research exploring collaboration in different scenarios, with different avatar types or MR setups.

Measurement of Empathy in Virtual Reality with Head-Mounted Displays: A Systematic Review.

We present a systematic review of 111 papers that measure the impact of virtual experiences created through head-mounted displays (HMDs) on empathy. Our goal was to analyze the conditions and the extent to which virtual reality (VR) enhances empathy. To achieve this, we categorized the relevant literature according to measurement methods, correlated human factors, viewing experiences, topics, and participants. Meta-analysis was performed based on categorized themes, and under specified conditions, we found that VR can improve empathy. Emotional empathy increased temporarily after the VR experience and returned to its original level over time, whereas cognitive empathy remained enhanced. Furthermore, while VR did not surpass 2D video in improving emotional empathy, it did enhance cognitive empathy, which is associated with embodiment. Our results are consistent with existing research suggesting differentiation between cognitive empathy (influenced by environmental factors and learnable) and emotional empathy (highly heritable and less variable). Interactivity, target of empathy, and point of view were not found to significantly affect empathy, but participants' age and nationality were found to influence empathy levels. It can be concluded that VR enhances cognitive empathy by immersing individuals in the perspective of others and that storytelling and personal characteristics are more important than the composition of the VR scene. Our findings provide guiding information for creating empathy content in VR and designing experiments to measure empathy.

Perceptual Thresholds for Radial Optic Flow Distortion in Near-Eye Stereoscopic Displays.

We provide the first perceptual quantification of user's sensitivity to radial optic flow artifacts and demonstrate a promising approach for masking this optic flow artifact via blink suppression. Near-eye HMOs allow users to feel immersed in virtual environments by providing visual cues, like motion parallax and stereoscopy, that mimic how we view the physical world. However, these systems exhibit a variety of perceptual artifacts that can limit their usability and the user's sense of presence in VR. One well-known artifact is the vergence-accommodation conflict (VAC). Varifocal displays can mitigate VAC, but bring with them other artifacts such as a change in virtual image size (radial optic flow) when the focal plane changes. We conducted a set of psychophysical studies to measure users' ability to perceive this radial flow artifact before, during, and after self-initiated blinks. Our results showed that visual sensitivity was reduced by a factor of 10 at the start and for ~70 ms after a blink was detected. Pre- and post-blink sensitivity was, on average, ~O.15% image size change during normal viewing and increased to ~1.5- 2.0% during blinks. Our results imply that a rapid (under 70 ms) radial optic flow distortion can go unnoticed during a blink. Furthermore, our results provide empirical data that can be used to inform engineering requirements for both hardware design and software-based graphical correction algorithms for future varifocal near-eye displays. Our project website is available at https://gamma.umd.edu/ROF/.

CAEVR: Biosignals-Driven Context-Aware Empathy in Virtual Reality.

There is little research on how Virtual Reality (VR) applications can identify and respond meaningfully to users' emotional changes. In this paper, we investigate the impact of Context-Aware Empathic VR (CAEVR) on the emotional and cognitive aspects of user experience in VR. We developed a real-time emotion prediction model using electroencephalography (EEG), electrodermal activity (EDA), and heart rate variability (HRV) and used this in personalized and generalized models for emotion recognition. We then explored the application of this model in a context-aware empathic (CAE) virtual agent and an emotion-adaptive (EA) VR environment. We found a significant increase in positive emotions, cognitive load, and empathy toward the CAE agent, suggesting the potential of CAEVR environments to refine user-agent interactions. We identify lessons learned from this study and directions for future work.

The Effects of Secondary Task Demands on Cybersickness in Active Exploration Virtual Reality Experiences.

Active exploration in virtual reality (VR) involves users navigating immersive virtual environments, going from one place to another. While navigating, users often engage in secondary tasks that require attentional resources, as in the case of distracted driving. Inspired by research generally studying the effects of task demands on cybersickness (CS), we investigated how the attentional demands specifically associated with secondary tasks performed during exploration affect CS. Downstream of this, we studied how increased attentional demands from secondary tasks affect spatial memory and navigational performance. We discuss the results of a multi-factorial between-subjects study, manipulating a secondary task's demand across two levels and studying its effects on CS in two different sickness-inducing levels of an exploration experience. The secondary task's demand was manipulated by parametrically varying $n$ in an aural $n$-back working memory task and the provocativeness of the experience was manipulated by varying how frequently users experienced a yaw-rotational reorientation effect during the exploration. Results revealed that increases in the secondary task's demand increased sickness levels, also resulting in a higher temporal onset rate, especially when the experience was not already highly sickening. Increased attentional demand from the secondary task also vitiated navigational performance and spatial memory. Overall, increased demands from secondary tasks performed during navigation produce deleterious effects on the VR experience.

Human Factors at Play: Understanding the Impact of Conditioning on Presence and Reaction Time in Mixed Reality.

A prerequisite to improving the presence of a user in mixed reality (MR) is the ability to measure and quantify presence. Traditionally, subjective questionnaires have been used to assess the level of presence. However, recent studies have shown that presence is correlated with objective and systemic human performance measures such as reaction time. These studies analyze the correlation between presence and reaction time when technical factors such as object realism and plausibility of the object's behavior change. However, additional psychological and physiological human factors can also impact presence. It is unclear if presence can be mapped to and correlated with reaction time when human factors such as conditioning are involved. To answer this question, we conducted an exploratory study ($N=60$) where the relationship between presence and reaction time was assessed under three different conditioning scenarios: control, positive, and negative. We demonstrated that human factors impact presence. We found that presence scores and reaction times are significantly correlated (correlation coefficient of -0.64), suggesting that the impact of human factors on reaction time correlates with its effect on presence. In demonstrating that, our study takes another important step toward using objective and systemic measures like reaction time as a presence measure.