Dissonance and Roughness in Cross-Modal Perception

Research output: Conference PapersRGC 32 - Refereed conference paper (without host publication)peer-review

View graph of relations

Author(s)

Detail(s)

Original languageEnglish
Number of pages5
Publication statusPublished - Aug 2017
Externally publishedYes

Conference

Title6th Conference of the Asia-Pacific Society for the Cognitive Sciences of Music (APSCOM 6)
LocationKyoto Women's University
PlaceJapan
CityKyoto
Period25 - 27 August 2017

Abstract

In music cognition, dissonance has been linked to musical features such as functional harmony and external contexts (e.g. working memory). However, researchers have also suggested that low-level psychoacoustic features, such as auditory roughness, are important to dissonance perception. Similarly, cross-modal mapping has been shown to be unconscious, and we posit that roughness and consequently dissonance perception may be involved in cross-modal processing. We conducted two studies examining the effects of perceived dissonance and computed roughness on cross-modal percepts. In Study 1, we explored possible linkages between visual and auditory constructs. Participants (N = 20) rated eight auditory stimuli for dissonance, perceived shape (curvy to spiky), texture (rough to smooth) and hardness (hard to soft) on a continuous scale. Each stimulus consisted of a stringed or flute instrument playing a single note, at a standard pitch. A linear mixed-effects regression showed significant effects of perceptual dissonance on texture (p = .002) and shape (p = .03). We employed MacCullum’s (2006) computational model to estimate auditory roughness, but no significant effects were found, possibly due to the narrow range of roughness values in instrumental sounds. Thus, in Study 2, another set of eight stimuli were generated, with four predetermined levels of roughness: none (0), low (0.1), medium (0.5) and high (1.0), across two fundamental pitches (110Hz and 220Hz). Roughness was manipulated through the adjustment of modulation index and harmonicity in frequency-modulation synthesis. Participants (N = 41) rated each stimulus for dissonance and undertook a two-alternative forced choice (2AFC) task, matching an image from a pair of graphics depicting a spiky object (previously rated as rough and hard) and a curvy object (smooth and soft) to the auditory stimulus. Findings from a logistic mixed-effects regression suggests that auditory roughness has a stronger effect on the graphic choice, with higher roughness linked to the spikier graphics (p < .001). No significant effects were found for dissonance. In Study 1 participants rated each audio stimulus along a verbal descriptive dimension, and with the exception of round-spiky, most of these items were conventionally used metaphors in describing timbre. However, in Study 2, participants performed a 2AFC between two visual objects different on these dimensions, and tested a direct audio-visual correspondence being sound and sight. As the two tests showed different patterns of effects, future investigations will need to clarify what is the discrete contribution of sensation and language to the cross modal experience of sound.

Research Area(s)

  • perception, sound, Cross-modal

Citation Format(s)

Dissonance and Roughness in Cross-Modal Perception. / Liew, Kongmeng; Styles, Suzy J.; Lindborg, PerMagnus.
2017. Paper presented at 6th Conference of the Asia-Pacific Society for the Cognitive Sciences of Music (APSCOM 6), Kyoto, Japan.

Research output: Conference PapersRGC 32 - Refereed conference paper (without host publication)peer-review