Speculative Auditory Virtual Environments: Evaluating and Extending the Artistic Possibilities of Tracking in Binaural Playback

Project: Research

View graph of relations


Binaural techniques reproduce sound as heard in one’s two ears and are intended for headphone playback, giving listeners a sense of being present at the recording location. This immersive effect originally developed in 1881 for telephone transmissions of opera (Paul 2009) is created through mimicking the acoustic effects of the human head and ears. Artists have created binaural recordings since 1978 (Reed 1978, Cardiff 1991). Since 1988, virtual environments have used binaural rendering with head-orientation tracking, causing corresponding rotations of soundfields as typical in the real world (Wenzel et al 1988). Since 2016, binaural rendering with tracking has become more affordable for creatives through its adoption into Virtual Reality formats.Despite this increase in accessibility, there are few artistic works specifically for binaural playback with tracking, and even fewer where the primary objective is not merely to locate users in visual virtual environments. The proposed research expands the use of binaural sound in VR beyond imitating real-world soundfields, developing the artistic potential of “creational” Auditory Virtual Environments (Novo 2005; Rumori and Marentakis 2017), or what we propose as “speculative” AVEs. We use the term “speculative” from design to mean imagining possible futures (Dunne and Raby 2013). Semi-structured interviews with eight artists based in Hong Kong and abroad and an online survey with 20 international participants will be conducted to determine the strategies used to create their existing works with binaural techniques and their intended effect. These two primary data sources will be coded and analysed to develop the first typology of binaural audio in art, of interest to musicology and sonic art scholars. The typology will be used by the PI to produce eight artistic works, each exemplifying a “signature”strategy of each interviewee and developed towards speculative AVEs for audience evaluation through focus groups. This systematic practice-based research uses an “iterative cyclic web of practice-led research and research-led practice” (Smith and Dean 2009) to extend the PI’s previous data sonification and visualisation work within immersive audiovisual environments (Ikeshiro 2012) to create speculative soundfields that would be impossible to experience in the real world, requiring a recalibration of headphone listening practice. Concurrently, we will contextualise the practice-based research through scholarly papers and develop an online best practice guide for artists and curators for designing and presenting binaural works. We will present the research and artistic outputs in an exhibition and a public showcase aimed at a general audience. 


Project number9048255
Grant typeECS
Effective start/end date1/01/23 → …