DESIGNING THE BALANCE BETWEEN SOUND AND TOUCH: METHODS FOR MULTIMODAL COMPOSITION
Abstract
In this paper, we discuss key research questions generatedfrom a collaborative workshop, during which our aim wasto explore the potential of a wearable device to producenovel audio-haptic sensory experiences. The main inten-tion of this research is to enable users with any type ofhearing profile to appreciate a body-centered listening ex-perience. The multimodal harness, developed by the co-authors, integrates nine voice coil actuators into a wear-able structure, stimulating both the auditory and tactilesenses via extra-tympanic sound conduction and vibrotac-tile stimulation of the skin on the upper body (spine, clavi-cles, ribs). We need to create our own interfaces in order tojudge its capacity to elicit three modes of vibratory soundperception: auditory, tactile and bi-modal. To this end,we used the Max environment to make several prelimi-nary authoring tools, whose compositional features allowus to explore three main themes of sensory composition:multimodal music listening experiences, spatialization ofaudio-haptic signals on the body, and sensory equaliza-tion. Feedback from trial sessions, along with currentconstraints due to the wearable and interface design, giveus direction for our future work: iterate to improve thesensory experience of the multimodal harness, and applythese tools experimentally in order to contribute to multi-sensory processing resear
Origin : Files produced by the author(s)