Design for the Felt-­Auditory Dimension: relationships, tools and strategies to sensitise somatic design practices to the felt-­effort qualities and features of walking on pressure-­mediated sound-­generating surfaces

Abstract

There is a growing interest in somatic approaches to support the design of movement-based technologies in Human-Computer Interaction. Somatic approaches, that focus on bodily movement awareness, can provide rich first-person accounts of movement experience to inform design. Recent research has used interactive sound as feedback on the qualities of movement to raise kinaesthetic awareness using somatic approaches. However, most of these approaches have concentrated on developing technologies and algorithms to extract movement features without really combining with the essence of movement from the felt, first-person perspective to inform design. This thesis proposes the Felt-Auditory dimension that combines first-person perspectives with movement data features to frame design processes. This thesis specifically examines walking with pressure-sensitive sound-generating surfaces and it asks two questions to examine the Felt-Auditory dimension, 1. What are suitable theories of movement and sound to facilitate an understanding of designing for movement-to-sound relationships? 2. What are the appropriate tools and strategies for designers to use in exploring the mappings between first-person felt efforts, computationally sensed data and sound synthesis to support kinaesthetic awareness? The Felt-Auditory dimension is both a theoretical position on the relationships made, and a sensitising resource for the practice of design for first-person movement-to-sound relationships. It consists of two main findings as a contribution to aesthetically motivated Sonic Interaction Design. The first finding is that Felt-Auditory relationships between walking movement on pressure mediated sound generating surfaces consist of emergent rhythms and continuous trajectories. The second is that Felt-Auditory tools and strategies require methods of visualisation to focus first-person perspectives with movement data features to frame design processes with sound synthesis and auditory display.