DDSP-Piano: a Neural Sound Synthesizer Informed by Instrument Knowledge - Institut de Recherche et Coordination Acoustique/Musique Access content directly
Journal Articles AES - Journal of the Audio Engineering Society Audio-Accoustics-Application Year : 2023

DDSP-Piano: a Neural Sound Synthesizer Informed by Instrument Knowledge


Instrument sound synthesis using deep neural networks has received numerous improvements over the last couple of years. Among them, the Differentiable Digital Signal Processing (DDSP) framework has modernized the spectral modeling paradigm by including signal-based synthesizers and effects into fully differentiable architectures. The present work extends the applications of DDSP to the task of polyphonic sound synthesis, with the proposal of a differentiable piano synthesizer conditioned on MIDI inputs. The model architecture is motivated by high-level acoustic modeling knowledge of the instrument, which, along with the sound structure priors inherent to the DDSP components, makes for a lightweight, interpretable, and realistic-sounding piano model. A subjective listening test has revealed that the proposed approach achieves better sound quality than a state-of-the-art neural-based piano synthesizer, but physical-modeling-based models still hold the best quality. Leveraging its interpretability and modularity, a qualitative analysis of the model behavior was also conducted: it highlights where additional modeling knowledge and optimization procedures could be inserted in order to improve the synthesis quality and the manipulation of sound properties. Eventually, the proposed differentiable synthesizer can be further used with other deep learning models for alternative musical tasks handling polyphonic audio and symbolic data.
Embargoed file
Embargoed file
0 6 21
Year Month Jours
Avant la publication

Dates and versions

hal-04073770 , version 1 (19-04-2023)
hal-04073770 , version 2 (15-09-2023)



Lenny Renault, Rémi Mignot, Axel Roebel. DDSP-Piano: a Neural Sound Synthesizer Informed by Instrument Knowledge. AES - Journal of the Audio Engineering Society Audio-Accoustics-Application, 2023, Special Issue on New Trends in Audio Effects, Part 2, 71 (9), pp.552-565. ⟨10.17743/jaes.2022.0102⟩. ⟨hal-04073770v1⟩


87 View
6 Download



Gmail Facebook Twitter LinkedIn More