Acoustic cues of keyboard mechanics enable auditory localization of upright piano tones.

Piano tone localization at the performer's listening point is a multisensory process involving audition, vision, and upper limb proprioception. The consequent representation of the auditory scene, especially in experienced pianists, is likely also influenced by their memory about the instrument keyboard. Disambiguating such components is not obvious, and first requires an analysis of the acoustic tone localization process to assess the role of auditory feedback in forming this scene. This analysis is complicated by the acoustic behavior of the piano, which does not guarantee the activation of the auditory precedence effect during a tone attack, nor can it provide robust interaural differences during the subsequent free evolution of the sound. In a tone localization task using a Disklavier upright piano (which can be operated remotely and configured to have its hammers hit a damper instead of producing a tone), twenty-three expert musicians, including pianists, successfully recognized the angular position of seven evenly distributed notes across the keyboard. The experiment involved listening to either full piano tones or just the key mechanical noise, with no additional feedback from other senses. This result suggests that the key mechanical noise alone activated the localization process without support from vision and/or limb proprioception. Since the same noise is present in the onset of the full tones, the key mechanics of our piano created a touch precursor in such tones that may be responsible of their correct angular localization by means of the auditory precedence effect. However, the significance of pitch cues arriving at a listener after the touch precursor was not measured when full tones were presented. As these cues characterize a note and, hence, the corresponding key position comprehensively, an open question remains regarding the contribution of pianists' spatial memory of the instrument keyboard to tone localization.

Modulating the Perceived Softness of Real Objects Through Wearable Feel-Through Haptics.

In vision, Augmented Reality (AR) allows the superposition of digital content on real-world visual information, relying on the well-established See-through paradigm. In the haptic domain, a putative Feel-through wearable device should allow to modify the tactile sensation without masking the actual cutaneous perception of the physical objects. To the best of our knowledge, a similar technology is still far to be effectively implemented. In this work, we present an approach that allows, for the first time, to modulate the perceived softness of real objects using a Feel-through wearable that uses a thin fabric as interaction surface. During the interaction with real objects, the device can modulate the growth of the contact area over the fingerpad without affecting the force experienced by the user, thus modulating the perceived softness. To this aim, the lifting mechanism of our system warps the fabric around the fingerpad in a way proportional to the force exerted on the specimen under exploration. At the same time, the stretching state of the fabric is controlled to keep a loose contact with the fingerpad. We demonstrated that different softness perceptions for the same specimens can be elicited, by suitably controlling the lifting mechanism of the system.

Altering Time Perception in Virtual Reality Through Multimodal Visual-Tactile Kappa Effect.

The perception of time is highly subjective and intertwined with space perception. In a well-known perceptual illusion, called Kappa effect, the distance between consecutive stimuli is modified to induce time distortions in the perceived inter-stimulus interval that are proportional to the distance between the stimuli. However, to the best of our knowledge, this effect has not been characterized and exploited in virtual reality (VR) within a multisensory elicitation framework. This paper investigates the Kappa effect elicited by concurrent visual-tactile stimuli delivered to the forearm, through a multimodal VR interface. This paper compares the outcomes of an experiment in VR with the results of the same experiment performed in the "physical world", where a multimodal interface was applied to participants' forearm to deliver controlled visual-tactile stimuli. Our results suggest that a multimodal Kappa effect can be elicited both in VR and in the physical world relying on concurrent visual-tactile stimulation. Moreover, our results confirm the existence of a relation between the ability of participants in discriminating the duration of time intervals and the magnitude of the experienced Kappa effect. These outcomes can be exploited to modulate the subjective perception of time in VR, paving the path toward more personalised human-computer interaction.

Flipping food during grilling tasks, a dataset of utensils kinematics and dynamics, food pose and subject gaze.

This paper presents a multivariate dataset of 2866 food flipping movements, performed by 4 chefs and 5 home cooks, with different grilled food and two utensils (spatula and tweezers). The 3D trajectories of strategic points in the utensils were tracked using optoelectronic motion capture. The pinching force of the tweezers, the bending force and torsion torque of the spatula were also recorded, as well as videos and the subject gaze. These data were collected using a custom experimental setup that allowed the execution of flipping movements with freshly cooked food, without having the sensors near the dangerous cooking area. Complementary, the 2D position of food was computed from the videos. The action of flipping food is, indeed, gaining the attention of both researchers and manufacturers of foodservice technology. The reported dataset contains valuable measurements (1) to characterize and model flipping movements as performed by humans, (2) to develop bio-inspired methods to control a cooking robot, or (3) to study new algorithms for human actions recognition.

An Open-Source Robotic Tool for the Simulation of Quasi-Static Finger Pressing on Stationary and Vibrating Surfaces.

The Bogus Finger is a remote-controllable tool for simulating vertical pressing forces of various magnitude as exerted by a human finger. Its main application is the characterization of haptic devices under realistic active touch conditions. The device is released as an open-source hardware and software DIY project that can be easily built using off-the-shelf components. We report the characterization of the quasi-static properties of the device, and validate its dynamic response to pressing on a vibrating surface by comparison with human fingers. The present prototype configuration accurately reproduces the mechanical impedance of the human finger in the frequency range 200-400 Hz.