In this paper we present ongoing work on the sonification of movements by dancers and players in Swedish folk music, with the aim to develop oral music theory tools for artistic and pedagogical purposes.
An advantage of using sonification in the exploration of dance and music interaction is that it places dance movements within the same sensory domain as music - sound. In general, human beings are more accurate in perceiving time differences with auditory than visual stimuli, and the ability to listen to the dance movements can facilitate a more precise understanding of the complex temporal relations between movements and music. Sonifying dance movements extend traditional music and dance practices into an artificially created sonic world. With sounding dance movements, the roles in the interaction of dancers and musicians become entangled, which can allow new ways of artistic expression.
This work aims at sonifying movement patterns in the dance in ways that 1) correspond to the embodied experience of the performers, 2) make the experience of how rhythms and meter interact in dancing and playing more tangible and, 3) allow for artistic explorations of performing with sonifications of dance.
As a first step we explore sonifying motion capture data of dancers and musicians performing together and sonify movements that are relevant to the rhythmic and metrical patterns of the music and dance forms. This initial focus on recorded data facilitates a sound design that involves first-person perspectives. To this end, we invite expert dancers and musicians to contribute to the design process. We sonify using WebaudioXML to facilitate accessible interactions in this process, through a web interface, and we will present the insights from our ongoing design process. In future extensions we aim to explore using real-time sensors to allow live interactions between sonified dancers and musicians.