Designing the Model Human Cochlea: An Ambient Crossmodal Audio-Tactile Display.

We present a model human cochlea (MHC), a sensory substitution technique and system that translates auditory information into vibrotactile stimuli using an ambient, tactile display. The model is used in the current study to translate music into discrete vibration signals displayed along the back of the body using a chair form factor. Voice coils facilitate the direct translation of auditory information onto the multiple discrete vibrotactile channels, which increases the potential to identify sections of the music that would otherwise be masked by the combined signal. One of the central goals of this work has been to improve accessibility to the emotional information expressed in music for users who are deaf or hard of hearing. To this end, we present our prototype of the MHC, two models of sensory substitution to support the translation of existing and new music, and some of the design challenges encountered throughout the development process. Results of a series of experiments conducted to assess the effectiveness of the MHC are discussed, followed by an overview of future directions for this research.

Effects of relative phases on pitch and timbre in the piano bass range.

Piano bass tones raise questions related to the perception of multicomponent, inharmonic tones. In this study, the influence of the relative phases among partials on pitch and timbre was investigated for synthesized bass tones with piano-like inharmonicity. Three sets of bass tones (A0 = 27.5 Hz, 100 partials, flat spectral envelope) were generated; harmonic, low inharmonic, and high inharmonic. For each set, five starting phase relations among partials were applied; sine phases, alternate (sine/cosine) phases, random phases, Schroeder phases, and negative Schroeder phases. The pitch and timbre of the tones were influenced markedly by the starting phases. Listening tests showed that listeners are able to discriminate between tones having different starting phase relations, and also that the pitch could be changed by manipulating the relative phases (octave, fifth, major third). A piano-like inharmonicity gives a characteristic randomizing effect of the phase relations over time in tones starting with nonrandom phase relations. A measure of the regularity of the phase differences between adjacent partials is suggested for quantifying this randomization process. The observed phase effects might be of importance in synthesizing, recording, and reproducing piano music.