A Comparison of Virtual Reality Menu Archetypes: Raycasting, Direct Input, and Marking Menus.

We contribute an analysis of the prevalence and relative performance of archetypal VR menu techniques. An initial survey of 108 menu interfaces in 84 popular commercial VR applications establishes common design characteristics. These characteristics motivate the design of raycast, direct, and marking menu archetypes, and a two-experiment comparison of their relative performance with one and two levels of hierarchy using 8 or 24 items. With a single-level menu, direct input is the fastest interaction technique in general, and is unaffected by number of items. With a two-level hierarchical menu, marking is fastest regardless of item number. Menus using raycasting, the most common menu interaction technique, were among the slowest of the tested menus but were rated most consistently usable. Using the combined results, we provide design and implementation recommendations with applications to general VR menu design.

Arm cooling selectively impacts sensorimotor control.

The benefits of cold have long been recognized in sport and medicine. However, it also brings costs, which have more rarely been investigated, notably in terms of sensorimotor control. We hypothesized that, in addition to peripheral effects, cold slows down the processing of proprioceptive cues, which has an impact on both feedback and feedforward control. We therefore compared the performances of participants whose right arm had been immersed in either cold water (arm temperature: 14°C) or lukewarm water (arm temperature: 34°C). In experiment 1, we administered a Fitts's pointing task and performed a kinematic analysis to determine whether sensorimotor control processes were affected by the cold. Results revealed 1) modifications in late kinematic parameters, suggesting changes in the use of proprioceptive feedback, and 2) modifications in early kinematic parameters, suggesting changes in action representations and/or feedforward processes. To explore our hypothesis further, we ran a second experiment in which no physical movement was involved, and thus no peripheral effects. Participants were administrated a hand laterality task, known to involve implicit motor imagery and assess the internal representation of the hand. They were shown left- and right-hand images randomly displayed in different orientations in the picture plane and had to identify as quickly and as accurately as possible whether each image was of the left hand or the right hand. Results revealed slower responses and more errors when participants had to mentally rotate the cooled hand in the extreme orientation of 160°, further suggesting the impact of cold on action representations.NEW & NOTEWORTHY We investigated how arm cooling modulates sensorimotor representations and sensorimotor control. Arm cooling induced changes in early kinematic parameters of pointing, suggesting an impact on feedforward processes or hand representation. Arm cooling induced changes in late kinematic parameters of pointing, suggesting an impact on feedback processes. Arm cooling also affected performance on a hand laterality task, suggesting that action representations were modified.

Use of variable online visual feedback to optimize sensorimotor coding and learning of a motor sequence.

The present study characterized the impact of reliable and/or unreliable online visual feedback and their order of presentation on the coding and learning of a motor sequence. Participants practiced a 12-element motor sequence 200 times. During this acquisition phase, two groups received a single type (i.e., either reliable or unreliable) of online visual feedback, two other groups encountered both types of feedback: either reliable first then unreliable, or unreliable first then reliable. Delayed retention tests and intermanual transfer tests (visuospatial and motor) were administered 24 hours later. Results showed that varying the reliability of online visual information during the acquisition phase allowed participants to use different task coding modalities without damaging their long-term sequence learning. Moreover, starting with reliable visual feedback, replaced halfway through with unreliable feedback promoted motor coding, which is seldom observed. This optimization of motor coding opens up interesting perspectives, as it is known to promote better learning of motor sequences.

Reliability of on-line visual feedback influences learning of continuous motor task of healthy young adults.

A continuous task was used to determine how the reliability of on-line visual feedback during acquisition impacts motor learning. Participants performed a right hand pointing task of a repeated sequence with a visual cursor that was either reliable, moderately unreliable, or largely unreliable. Delayed retention tests were administered 24 h later, as well as intermanual transfer tests (performed with the left hand). A visuospatial transfer test was performed with the same targets' sequence (same visuospatial configuration) while a motor transfer test was performed with the visual mirror of the targets' sequence (same motor patterns). Results showed that pointing was slower and long-term learning disrupted in the largely unreliable visual cursor condition, compared with the reliable and moderately unreliable conditions. Also, analysis of transfers revealed classically better performance on visuospatial transfer than on motor transfer for the reliable condition. However, here we first show that such difference disappears when the cursor was moderately or largely unreliable. Interestingly, these results indicated a difference in the type of sequence coding, depending on the reliability of the on-line visual feedback. This recourse to mixed coding opens up interesting perspectives, as it is known to promote better learning of motor sequences.

Studying the timescale of perceptual-motor (re)calibration following a change in visual display gain.

Experiencing a non-1:1 mapping between perception and action in everyday life is not common. It could be considered as a problem for our perceptual-motor system because of the need to adapt our goal-directed movement to different gains between movement and task spaces. In the Human Computer Interface domain, the main example of such a situation consists in switching from one operating system to another which requires to adapt our movement to different Control Display gains. The aim of the study was to characterize the perceptual-motor calibration process following a sudden change in control display gain. Sixteen participants manipulated a mouse computer to move a cursor on the screen. The discrete aiming task consisted on reaching the target from a starting target position as fast and as accurately as possible. Our methodology consisted in suddenly manipulating the gain between both spaces following a three-step adaptation methodology (baseline condition followed by a perturbation and return to baseline condition). Results demonstrated that not only participants produce adaptive behavior following several types of perturbations, but they were also able to do it at a very short timescale. As the calibration process described in the present study may play a significant role in the acquisition of accurate perceptual-motor skills involving the use of devices that augment human fine motor capabilities (e.g., telesurgery, mouse and joystick), we conclude that this study could have important implications in the domain of Human-Computer Interaction (HCI) as well as in the domain Human Equipment Interaction.

Short-Term Sensorimotor Deprivation Impacts Feedforward and Feedback Processes of Motor Control.

Sensory loss involves irreversible behavioral and neural changes. Paradigms of short-term limb immobilization mimic deprivation of proprioceptive inputs and motor commands, which occur after the loss of limb use. While several studies have shown that short-term immobilization induced motor control impairments, the origin of such modifications is an open question. A Fitts' pointing task was conducted, and kinematic analyses were performed to assess whether the feedforward and/or feedback processes of motor control were impacted. The Fitts' pointing task specifically required dealing with spatial and temporal aspects (speed-accuracy trade-off) to be as fast and as accurate as possible. Forty trials were performed on two consecutive days by Control and Immobilized participants who wore a splint on the right arm during this 24 h period. The immobilization modified the motor control in a way that the full spatiotemporal structure of the pointing movements differed: A global slowdown appeared. The acceleration and deceleration phases were both longer, suggesting that immobilization impacted both the early impulse phase based on sensorimotor expectations and the later online correction phase based on feedback use. First, the feedforward control may have been less efficient, probably because the internal model of the immobilized limb would have been incorrectly updated relative to internal and environmental constraints. Second, immobilized participants may have taken more time to correct their movements and precisely reach the target, as the processing of proprioceptive feedback might have been altered.

Sensorimotor control and linear visuomotor gains.

In everyday life, we often use graphical interfaces where the visual space is mapped to the motor space with a visuomotor gain called the control display gain. One of the key objectives in the field of Human Computer Interaction is to design this control display gain so as to enhance users' performance. Although the control display gain involved in operating systems has been found to improve users' pointing performance, the reasons for this improvement have not yet been fully elucidated, especially because the control display gains on operating systems are both non-constant and non-linear. Here, we tested non-constant but linear velocity-based control display gains to determine which parameters were responsible for pointing performance changes based on analyses of the movement kinematics. Using a Fitts' paradigm, constant gains of 1 and 3 were compared with a linearly increasing gain (i.e., the control display gain increases with the motor velocity) and a decreasing gain (i.e., the control display gain decreases with the motor velocity). Three movements with various indexes of difficulty (ID) were tested (3, 5 and 7 bits). The increasing gain was expected to increase the velocity of the initial impulse phase and decrease that of the correction phase, thus decreasing the movement time (MT), and the contrary in the case of the decreasing gain. Although the decreasing gain increased MT at ID3, the increasing gain was found to be less efficient than the constant gain of 3, probably because a non-constant gain between the motion and its visual consequences disrupted the sensorimotor control. In addition, the kinematic analyses of the movements suggested that the motion profile was planned by the central nervous system based on the visuomotor gain at maximum motor velocity, as common features were observed between the constant gain of 1 and the decreasing gain, and between the constant gain of 3 and the increasing gain. By contrast, the amplitude of the velocity profile seemed to be specific to each particular visuomotor mapping process.

Towards BCI-Based Interfaces for Augmented Reality: Feasibility, Design and Evaluation.

Brain-Computer Interfaces (BCIs) enable users to interact with computers without any dedicated movement, bringing new hands-free interaction paradigms. In this paper we study the combination of BCI and Augmented Reality (AR). We first tested the feasibility of using BCI in AR settings based on Optical See-Through Head-Mounted Displays (OST-HMDs). Experimental results showed that a BCI and an OST-HMD equipment (EEG headset and Hololens in our case) are well compatible and that small movements of the head can be tolerated when using the BCI. Second, we introduced a design space for command display strategies based on BCI in AR, when exploiting a famous brain pattern called Steady-State Visually Evoked Potential (SSVEP). Our design space relies on five dimensions concerning the visual layout of the BCI menu; namely: orientation, frame-of-reference, anchorage, size and explicitness. We implemented various BCI-based display strategies and tested them within the context of mobile robot control in AR. Our findings were finally integrated within an operational prototype based on a real mobile robot that is controlled in AR using a BCI and a HoloLens headset. Taken together our results (4 user studies) and our methodology could pave the way to future interaction schemes in Augmented Reality exploiting 3D User Interfaces based on brain activity and BCIs.

Design and Evaluation of Fusion Approach for Combining Brain and Gaze Inputs for Target Selection.

Gaze-based interfaces and Brain-Computer Interfaces (BCIs) allow for hands-free human-computer interaction. In this paper, we investigate the combination of gaze and BCIs. We propose a novel selection technique for 2D target acquisition based on input fusion. This new approach combines the probabilistic models for each input, in order to better estimate the intent of the user. We evaluated its performance against the existing gaze and brain-computer interaction techniques. Twelve participants took part in our study, in which they had to search and select 2D targets with each of the evaluated techniques. Our fusion-based hybrid interaction technique was found to be more reliable than the previous gaze and BCI hybrid interaction techniques for 10 participants over 12, while being 29% faster on average. However, similarly to what has been observed in hybrid gaze-and-speech interaction, gaze-only interaction technique still provides the best performance. Our results should encourage the use of input fusion, as opposed to sequential interaction, in order to design better hybrid interfaces.

Integrality and separability of multitouch interaction techniques in 3D manipulation tasks.

Multitouch displays represent a promising technology for the display and manipulation of data. While the manipulation of 2D data has been widely explored, 3D manipulation with multitouch displays remains largely unexplored. Based on an analysis of the integration and separation of degrees of freedom, we propose a taxonomy for 3D manipulation techniques with multitouch displays. Using that taxonomy, we introduce Depth-Separated Screen-Space (DS3), a new 3D manipulation technique based on the separation of translation and rotation. In a controlled experiment, we compared DS3 with Sticky Tools and Screen-Space. Results show that separating the control of translation and rotation significantly affects performance for 3D manipulation, with DS3 performing faster than the two other techniques.

Depth-of-field blur effects for first-person navigation in virtual environments.

Depth-of-field blur effects are well-known depth cues in human vision. Computer graphics pipelines added DOF effects early to enhance imagery realism, but real-time VR applications haven't yet introduced visual blur effects. The authors describe new techniques to improve blur rendering and report experimental results from a prototype video game implementation.