Data augmentation for generating synthetic electrogastrogram time series.

To address an emerging need for large number of diverse datasets for rigor evaluation of signal processing techniques, we developed and evaluated a new method for generating synthetic electrogastrogram time series. We used electrogastrography (EGG) data from an open database to set model parameters and statistical tests to evaluate synthesized data. Additionally, we illustrated method customization for generating artificial EGG time series alterations caused by the simulator sickness. Proposed data augmentation method generates synthetic EGG data with specified duration, sampling frequency, recording state (postprandial or fasting state), overall noise and breathing artifact injection, and pauses in the gastric rhythm (arrhythmia occurrence) with statistically significant difference between postprandial and fasting states in > 70% cases while not accounting for individual differences. Features obtained from the synthetic EGG signal resembling simulator sickness occurrence displayed expected trends. The code for generation of synthetic EGG time series is not only freely available and can be further customized to assess signal processing algorithms but also may be used to increase data diversity for training artificial intelligence (AI) algorithms. The proposed approach is customized for EGG data synthesis but can be easily utilized for other biosignals with similar nature such as electroencephalogram.

Effectiveness of a time to fixate for fitness to drive evaluation in neurological patients.

We present a method to automatically calculate time to fixate (TTF) from the eye-tracker data in subjects with neurological impairment using a driving simulator. TTF presents the time interval for a person to notice the stimulus from its first occurrence. Precisely, we measured the time since the children started to cross the street until the drivers directed their look to the children. From 108 neurological patients recruited for the study, the analysis of TTF was performed in 56 patients to assess fit-, unfit-, and conditionally-fit-to-drive patients. The results showed that the proposed method based on the YOLO (you only look once) object detector is efficient for computing TTFs from the eye-tracker data. We obtained discriminative results for fit-to-drive patients by application of Tukey's honest significant difference post hoc test (p < 0.01), while no difference was observed between conditionally-fit and unfit-to-drive groups (p = 0.542). Moreover, we show that time-to-collision (TTC), initial gaze distance (IGD) from pedestrians, and speed at the hazard onset did not influence the result, while the only significant interaction is among fitness, IGD, and TTC on TTF. Obtained TTFs are also compared with the perception response times (PRT) calculated independently from eye-tracker data and YOLO. Although we reached statistically significant results that speak in favor of possible method application for assessment of fitness to drive, we provide detailed directions for future driving simulation-based evaluation and propose processing workflow to secure reliable TTF calculation and its possible application in for example psychology and neuroscience.

Electrogastrogram-Derived Features for Automated Sickness Detection in Driving Simulator.

The rapid development of driving simulators for the evaluation of automated driving experience is constrained by the simulator sickness-related nausea. The electrogastrogram (EGG)-based approach may be promising for immediate, objective, and quantitative nausea assessment. Given the relatively high EGG sensitivity to noises associated with the relatively low amplitude and frequency spans, we introduce an automated procedure comprising statistical analysis and machine learning techniques for EGG-based nausea detection in relation to the noise contamination during automated driving simulation. We calculate the root mean square of EGG amplitude, median and dominant frequencies, magnitude of Power Spectral Density (PSD) at dominant frequency, crest factor of PSD, and spectral variation distribution along with newly introduced parameters: sample and spectral entropy, autocorrelation zero-crossing, and parameters derived from the Poincaré diagram of consecutive EGG samples. Results showed outstanding robustness of sample entropy with moderate robustness of autocorrelation zero-crossing, dominant frequency, and its median. Machine learning reached an accuracy of 88.2% and revealed sample entropy as one of the most relevant and robust parameters, while linear analysis highlighted spectral entropy, spectral variation distribution, and crest factor of PSD. This study clearly indicates the need for customized feature selection in noisy environments, as well as a complementary approach comprising machine learning and statistical analysis for efficient nausea detection.

A user study of directional tactile and auditory user interfaces for take-over requests in conditionally automated vehicles.

General introduction of unconditionally and conditionally automated vehicles is expected to have a highly positive impact on the society, from increased accessibility to mobility and road traffic safety, to decreased environmental and economic negative impacts. However, there are several obstacles and risks slowing down the adoption of this technology, which are primarily related to the human-machine interaction (HMI) and exchange of control between the vehicle and the human driver. In this article, we present key takeaways for HMI design of take-over requests (TOR) that the vehicle issues to inform the driver to take over control of the vehicle. The key takeaways were developed based on the results of a user study, where directional tactile-ambient (visual) and auditory-ambient (visual) TOR user interfaces (UI) were evaluated with regards to commonly used take-over quality aspects (attention redirection, take-over time, correct interpretation of stimuli, off-road drive, brake application, lateral acceleration, minimal time-to-collision and occurrence of collision). 36 participants took part in the mixed design study, which was conducted in a driving simulator. The results showed that drivers' attention was statistically significantly faster redirected with the auditory-ambient UI, however using the tactile-ambient UI resulted in less off-road driving and slightly less collisions. The results also revealed that drivers correctly interpreted the directional TOR stimuli more often than the non-directional one. Based on the study results, a list of key takeaways was developed and is presented in the conclusion of the paper. The results from this study are especially relevant to the TOR UI designers and the automotive industry, which tend to provide the most usable UI for ensuring safer end efficient human-vehicle interaction during the TOR task.

Consistency of neuropsychological and driving simulator assessment after neurological impairment.

Deficits in attentional and executive functioning may interfere with driving ability and result in a lower level of fitness to drive. Studies show mixed results in relation to the consistency of neuropsychological and driving simulator assessment. The objective of this study was to investigate the consistency of both types of assessment. Ninety-nine patients with various neurological impairments (72 males; M = 48.98 years; SD = 17.27) performed a 30-minute drive in a driving simulation in three different road settings; a (non-)residential rural area, a highway and an urban area. They also underwent neuropsychological assessment of attention and executive function. An exploratory correlational analysis was conducted. We found weak, but significant correlations between attention and executive function measures and more efficient driving in the driving simulator. Distractibility was associated with the most simulator variables in all three simulated road settings. Participants who were better at maintaining attention, eliminating irrelevant information and suppressing inappropriate responses, were less likely to drive above the speed limit, produced a less jerky ride, and used the rearview mirror more regularly. A lack of moderate or strong significant correlations (inconsistency) between traditional neuropsychological and simulator assessment variables may indicate that they don't evaluate the same cognitive processes.

A safety score for the assessment of driving style.

OBJECTIVE: Road traffic laws explicitly refer to a safe and cautious driving style as a means of ensuring safety. For automated vehicles to adhere to these laws, objective measurements of safe and cautious behavior in normal driving conditions are required. This paper describes the conception, implementation and initial testing of an objective scoring system that assigns safety indexes to observed driving style, and aggregates them to provide an overall safety score for a given driving session. METHODS: The safety score was developed by matching safety indexes with maneuver-based parameter ranges processed from an existing highway traffic data set with a newly developed algorithm. The concept stands on the idea that safety, rather than suddenly changing from a safe to an unsafe condition at a certain parameter value, can be better modeled as a continuum of values that consider the safety margins available for interactions among multiple vehicles and that depend on present traffic conditions. A sensitivity test of the developed safety score was conducted by comparing the results of applying the algorithm to two drivers in a simulator who were instructed to drive normally and risky, respectively. RESULTS: The evaluation of normal driving statistics provided suitable ranges for safety parameters like vehicle distances, time headways, and time to collision based on real traffic data. The sensitivity test provided preliminary evidence that the scoring method can discriminate between safe and risky drivers based on their driving style. In contrast to previous approaches, collision situations are not needed for this assessment. CONCLUSIONS: The developed safety score shows potential for assessing the level of safety of automated vehicle (AV) behavior in traffic, including AV ability to avoid exposure to collision-prone situations. Occasional bad scores may occur even for good drivers or autonomously driving vehicles. However, if the safety index becomes low during a significant part of a driving session, due to frequent or harsh safety margin violations, the corresponding driving style should not be accepted for driving in real traffic.

Electrogastrography in Autonomous Vehicles-An Objective Method for Assessment of Motion Sickness in Simulated Driving Environments.

Autonomous vehicles are expected to take complete control of the driving process, enabling the former drivers to act as passengers only. This could lead to increased sickness as they can be engaged in tasks other than driving. Adopting different sickness mitigation techniques gives us unique types of motion sickness in autonomous vehicles to be studied. In this paper, we report on a study where we explored the possibilities of assessing motion sickness with electrogastrography (EGG), a non-invasive method used to measure the myoelectric activity of the stomach, and its potential usage in autonomous vehicles (AVs). The study was conducted in a high-fidelity driving simulator with a virtual reality (VR) headset. There separate EGG measurements were performed: before, during and after the driving AV simulation video in VR. During the driving, the participants encountered two driving environments: a straight and less dynamic highway road and a highly dynamic and curvy countryside road. The EGG signal was recorded with a proprietary 3-channel recording device and Ag/AgCl cutaneous electrodes. In addition, participants were asked to signalize whenever they felt uncomfortable and nauseated by pressing a special button. After the drive they completed also the Simulator Sickness Questionnaire (SSQ) and reported on their overall subjective perception of sickness symptoms. The EGG results showed a significant increase of the dominant frequency (DF) and the percentage of the high power spectrum density (FSD) as well as a significant decrease of the power spectrum density Crest factor (CF) during the AV simulation. The vast majority of participants reported nausea during more dynamic conditions, accompanied by an increase in the amplitude and the RMS value of EGG. Reported nausea occurred simultaneously with the increase in EGG amplitude. Based on the results, we conclude that EGG could be used for assessment of motion sickness in autonomous vehicles. DF, CF and FSD can be used as overall sickness indicators, while the relative increase in amplitude of EGG signal and duration of that increase can be used as short-term sickness indicators where the driving environment may affect the driver.

Exploring driving characteristics of fit- and unfit-to-drive neurological patients: a driving simulator study.

Objective: To identify driving characteristics of fit-, unfit-, and conditionally fit-to-drive neurological patient populations using a driving simulator with three high-risk scenarios comprising rural, highway, and urban environments.Methods: The study included 91 neurological patients undergoing a multidisciplinary assessment for driver's license revalidation, consisting of a clinical, neuropsychological, functional, and on-road evaluation. The groups drove through three independent driving scenarios, during which a variety of measures describing reaction time, vehicular control, and traffic rule compliance were performed. One-way analysis of variance (ANOVA) with Bonferroni correction was used for group comparison, independently for each driving scenario, and Pearson correlations were calculated between simulator variables and neuropsychological test scores.Results: The fit- and unfit-to-drive population significantly differed (p < .05) in reaction times, regardless of the scenario. No significant differences in traffic rule compliance or vehicular control parameters were observed in the rural environment (p > .05). On the highway, the unfit group exhibited greater variability of steering wheel angle, higher steering reversal rate, and a higher rate of turn signal errors. In the urban environment, the unfit group oversped more, had more collisions, and exhibited greater lane position variability. The latter, along with reaction times in the rural and highway scenarios, was also shown to significantly differ between the conditional and unfit group (p < .05). No significant differences were observed between the fit and the conditional group (p > .05). Weak to moderate associations (range: -0.5 to 0.29) between neuropsychological tests and various simulator variables were also observed.Conclusions: Our results show that driving simulators are able to capture differences between (fit- and unfit-to-drive) neurological patient populations and therefore bear the potential for being used as a deficit-independent screening, assessment, or rehabilitation tool. The conditionally-fit-to-drive group exhibited less discriminative features, which points to greater importance of human judgment for this population. The observation that differences in most of the parameters were environment-dependent suggests that developers of future driver simulation tools should carefully design scenarios in order to fully exploit their assessment potential.

Lessons Learned: Gastric Motility Assessment During Driving Simulation.

In the era of technological advances and innovations in transportation technologies, application of driving simulators for the investigation and assessment of the driving process provides a safe and suitable testing environment. Although driving simulators are crucial for further improvements in transportation, it is important to resolve one of their main disadvantages-simulator sickness. Therefore, suitable methods for the assessment of simulator sickness are required. The main aim of this paper was to present a non-invasive method for assessing simulator sickness by recording gastric myoelectrical activity-electrogastrography. Open-source hardware for electrogastrography together with recordings obtained in 13 healthy volunteers is presented, and the main aspects of signal processing for artifact cancellation and feature extraction were discussed. Based on the obtained results, it was concluded that slow-wave electrical gastric activity can be recorded during driving simulation by following adequate recommendations and that proposed features could be beneficial in describing non-ordinary electrogastrography signals.

Validation of auditory detection response task method for assessing the attentional effects of cognitive load.

OBJECTIVES: There are 3 standardized versions of the Detection Response Task (DRT), 2 using visual stimuli (remote DRT and head-mounted DRT) and one using tactile stimuli. In this article, we present a study that proposes and validates a type of auditory signal to be used as DRT stimulus and evaluate the proposed auditory version of this method by comparing it with the standardized visual and tactile version. METHODS: This was a within-subject design study performed in a driving simulator with 24 participants. Each participant performed 8 2-min-long driving sessions in which they had to perform 3 different tasks: driving, answering to DRT stimuli, and performing a cognitive task (n-back task). Presence of additional cognitive load and type of DRT stimuli were defined as independent variables. DRT response times and hit rates, n-back task performance, and pupil size were observed as dependent variables. RESULTS: Significant changes in pupil size for trials with a cognitive task compared to trials without showed that cognitive load was induced properly. Each DRT version showed a significant increase in response times and a decrease in hit rates for trials with a secondary cognitive task compared to trials without. Similar and significantly better results in differences in response times and hit rates were obtained for the auditory and tactile version compared to the visual version. There were no significant differences in performance rate between the trials without DRT stimuli compared to trials with and among the trials with different DRT stimuli modalities. CONCLUSIONS: The results from this study show that the auditory DRT version, using the signal implementation suggested in this article, is sensitive to the effects of cognitive load on driver's attention and is significantly better than the remote visual and tactile version for auditory-vocal cognitive (n-back) secondary tasks.

An analysis of the suitability of a low-cost eye tracker for assessing the cognitive load of drivers.

This paper presents a driving simulator study in which we investigated whether the Eye Tribe eye tracker (ET) is capable of assessing changes in the cognitive load of drivers through oculography and pupillometry. In the study, participants were asked to drive a simulated vehicle and simultaneously perform a set of secondary tasks with different cognitive complexity levels. We measured changes in eye properties, such as the pupil size, blink rate and fixation time. We also performed a measurement with a Detection Response Task (DRT) to validate the results and to prove a steady increase of cognitive load with increasing secondary task difficulty. The results showed that the ET precisely recognizes an increasing pupil diameter with increasing secondary task difficulty. In addition, the ET shows increasing blink rates, decreasing fixation time and narrowing of the attention field with increasing secondary task difficulty. The results were validated with the DRT method and the secondary task performance. We conclude that the Eye Tribe ET is a suitable device for assessing a driver's cognitive load.

Detection-Response Task-Uses and Limitations.

The Detection-Response Task is a method for assessing the attentional effects of cognitive load in a driving environment. Drivers are presented with a sensory stimulus every 3-5 s, and are asked to respond to it by pressing a button attached to their finger. Response times and hit rates are interpreted as indicators of the attentional effect of cognitive load. The stimuli can be visual, tactile and auditory, and are chosen based on the type of in-vehicle system or device that is being evaluated. Its biggest disadvantage is that the method itself also affects the driver's performance and secondary task completion times. Nevertheless, this is an easy to use and implement method, which allows relevant assessment and evaluation of in-vehicle systems. By following the recommendations and taking into account its limitations, researchers can obtain reliable and valuable results on the attentional effects of cognitive load on drivers.

On the selection of stimulus for the auditory variant of the detection response task method for driving experiments.

OBJECTIVES: The detection response task (DRT) is a method for measuring attentional effects of secondary tasks on a driver's cognitive load by measuring response times and hit rates to different types of stimuli as indirect indicators of increased cognitive load. ISO 17488 (International Organization for Standardization 2016) only provides guidelines for the technical implementation and measurement methods for the visual and tactile versions (use of visual and tactile stimuli) of the DRT method. This article presents a study with the goal of finding the most appropriate auditory stimulus for the implementation of an auditory version of the DRT method. METHODS: This article presents the results of an experiment in which responses to 7 different auditory DRT stimuli-varying in frequency-were compared while inducing users' cognitive load with a modified n-back task. The experiment was conducted in a surrogate driving environment and in a within-subject design. Response times, hit rates, and secondary task performances were observed as indicators of increased cognitive load. RESULTS: Significantly shorter response times were found for the white noise signal compared to single-frequency signals. However, the largest differences in response times, for trials without and with a cognitive task, were found for 4- and 8-kHz single-frequency signals. No significant differences were found for hit rates and secondary task performances between the different stimuli. CONCLUSIONS: Consistent significant differences in response times for all tested stimuli prove that the auditory DRT variant is also sensitive to changes in cognitive load. The mean increase in response times of more than 25% for 4- and 8-kHz signals for trials with a cognitive task compared to trials without one indicates that one of these signals could be used as a potential auditory stimulus for the auditory DRT variant.

Sensitivity evaluation of the visual, tactile, and auditory detection response task method while driving.

OBJECTIVES: In this article, we evaluate the sensitivity to cognitive load of 3 versions of the Detection Response Task method (DRT), proposed in ISO Draft Standard DIS-17488. METHODS: We present a user study with 30 participants in which we compared the sensitivity to cognitive load of visual, audio, and tactile DRT in a simulated driving environment. The amount of cognitive load was manipulated with secondary n-back tasks at 2 levels of difficulty (0-back and 1-back). We also explored whether the DRT method is least sensitive to cognitive load when the stimuli and secondary task are of the same modality. For this purpose, we used 3 forms to present the n-back task stimuli: visual, audio, and tactile. Responses to the task were always vocal. The experiment was based on a between-subject design (the DRT modalities) with 2 levels of within-subject design study (modalities and difficulty of the secondary n-back tasks). The participants' primary task in the study was to drive safely, and a second priority was to answer to DRT stimuli and perform secondary tasks. RESULTS: The results indicate that all 3 versions of the DRT tested were sensitive to detecting the difference in cognitive load between the reference driving period and driving and engaging in the secondary tasks. Only the visual DRT discriminated between the 0-back and 1-back conditions on mean response time. Contrary to expectations, no interaction was observed between DRT modality and the stimuli modality used for presentation of the secondary tasks. CONCLUSIONS: None of the 3 methods of presenting DRT stimuli showed a consistent advantage in sensitivity in differentiating multiple levels of cognitive load if all response times, hit rates, and secondary task performance are considered. If only response time is considered, the visual presentation of the DRT stimulus used in this study showed some advantages. In interpreting these data, it should be noted that the methods of DRT stimulus presentation varied somewhat from the currently proposed draft ISO standard and it is possible that the relative salience level of the visual DRT stimulus influenced the findings. It is further suggested that more than 2 levels of difficulty of the n-back task should be considered for further investigation of the relative sensitivity of different DRT stimuli modalities. Parameters that indicate change in cognitive load (response time, hit rate, task performance) should be analyzed together in assessing the overall impact on the driver and not individually, in order to obtain a fuller insight of the assessed cognitive load.

Applications of Paper and Interactive Prototypes in Designing Telecare Services for Older Adults.

We present the results of the user studies of the CareMe prototype telecare service for supporting older adults with remote telecare services. The CareMe solution targets older adults who are capable of living independently and outside of a dedicated medical institution or a nursing home, but require occasional medical attention, and formal or informal care. The studies were conducted between November 2013 and March 2014 and followed the participatory design approach. The research questions focused on understanding the difference in the participatory design approach when older adults use paper prototypes vs interactive prototypes, and the difference when older adults use interactive prototypes with poor interface design vs interactive prototypes with rich interface design. In total, 45 older adults living in Slovenia participated in the field trial. They were retired, 32 of whom were nursing home residents and 13 were living independently at home. The study sample's age distribution was between 64 and 91 years. The analysis of the filed trial results indicated the importance of the factors that influence users' decision about how and when they use new technology, i.e., the perceived usefulness and the perceived ease-of-use. Main insights gained from the studies show, that it is very difficult to keep older people focused on the topics of discussion and that they have often difficulties to clearly present/express their ideas. Furthermore, abstract concepts, such as paper only prototypes proved to be challenging to cope with, whereas using real applications and services when trying out scenarios using interactive prototypes provided much better experience and feedback.

Exploring direct 3D interaction for full horizontal parallax light field displays using leap motion controller.

This paper reports on the design and evaluation of direct 3D gesture interaction with a full horizontal parallax light field display. A light field display defines a visual scene using directional light beams emitted from multiple light sources as if they are emitted from scene points. Each scene point is rendered individually resulting in more realistic and accurate 3D visualization compared to other 3D displaying technologies. We propose an interaction setup combining the visualization of objects within the Field Of View (FOV) of a light field display and their selection through freehand gesture tracked by the Leap Motion Controller. The accuracy and usefulness of the proposed interaction setup was also evaluated in a user study with test subjects. The results of the study revealed high user preference for free hand interaction with light field display as well as relatively low cognitive demand of this technique. Further, our results also revealed some limitations and adjustments of the proposed setup to be addressed in future work.

A user study of auditory, head-up and multi-modal displays in vehicles.

This paper describes a user study on the interaction with an in-vehicle information system (IVIS). The motivation for conducting this research was to investigate the subjectively and objectively measured impact of using a single- or multi-modal IVIS while driving. A hierarchical, list-based menu was presented using a windshield projection (head-up display), auditory display and a combination of both interfaces. The users were asked to navigate a vehicle in a driving simulator and simultaneously perform a set of tasks of varying complexity. The experiment showed that the interaction with visual and audio-visual head-up displays is faster and more efficient than with the audio-only display. All the interfaces had a similar impact on the overall driving performance. There was no significant difference between the visual only and audio-visual displays in terms of their efficiency and safety; however, the majority of test subjects clearly preferred to use the multi-modal interface while driving.

An analysis of the precision and reliability of the leap motion sensor and its suitability for static and dynamic tracking.

We present the results of an evaluation of the performance of the Leap Motion Controller with the aid of a professional, high-precision, fast motion tracking system. A set of static and dynamic measurements was performed with different numbers of tracking objects and configurations. For the static measurements, a plastic arm model simulating a human arm was used. A set of 37 reference locations was selected to cover the controller's sensory space. For the dynamic measurements, a special V-shaped tool, consisting of two tracking objects maintaining a constant distance between them, was created to simulate two human fingers. In the static scenario, the standard deviation was less than 0.5 mm. The linear correlation revealed a significant increase in the standard deviation when moving away from the controller. The results of the dynamic scenario revealed the inconsistent performance of the controller, with a significant drop in accuracy for samples taken more than 250 mm above the controller's surface. The Leap Motion Controller undoubtedly represents a revolutionary input device for gesture-based human-computer interaction; however, due to its rather limited sensory space and inconsistent sampling frequency, in its current configuration it cannot currently be used as a professional tracking system.