Explainable Audio Classification of Playing Techniques with Layer-wise Relevance Propagation - Laboratoire Traitement et Communication de l'Information Access content directly
Conference Papers Year : 2023

Explainable Audio Classification of Playing Techniques with Layer-wise Relevance Propagation

Abstract

Deep convolutional networks (convnets) in the time-frequency domain can learn an accurate and fine-grained categorization of sounds. For example, in the context of music signal analysis, this categorization may correspond to a taxonomy of playing techniques: vibrato, tremolo, trill, and so forth. However, convnets lack an explicit connection with the neurophysiological underpinnings of musical timbre perception. In this article, we propose a data-driven approach to explain audio classification in terms of physical attributes in sound production. We borrow from current literature in "explainable AI" (XAI) to study the predictions of a convnet which achieves an almost perfect score on a challenging task: i.e., the classification of five comparable real-world playing techniques from 30 instruments spanning seven octaves. Mapping the signal into the carrier-modulation domain using scattering transform, we decompose the networks' predictions over this domain with layer-wise relevance propagation. We find that regions highly-relevant to the predictions localized around the physical attributes with which the playing techniques are performed.
Fichier principal
Vignette du fichier
wang_ICASSP23_final.pdf (471.91 Ko) Télécharger le fichier
Origin : Files produced by the author(s)
licence : CC BY - Attribution

Dates and versions

hal-04029145 , version 1 (14-03-2023)

Licence

Attribution

Identifiers

Cite

Changhong Wang, Vincent Lostanlen, Mathieu Lagrange. Explainable Audio Classification of Playing Techniques with Layer-wise Relevance Propagation. 2023 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Jun 2023, Rhodes, Greece. pp.1-5, ⟨10.1109/ICASSP49357.2023.10095894⟩. ⟨hal-04029145⟩
215 View
188 Download

Altmetric

Share

Gmail Facebook X LinkedIn More