Timbre-enhanced Multi-modal Music Style Transfer with Domain Balance Loss

Tsai Jyun Fan, Chien Yu Lu, Wei Chen Chiu, Li Su, Che Rung Lee

研究成果: Conference contribution同行評審

摘要

Style transfer of the polyphonic music recordings has always been a challenging task due to the difficulty of learning representations for both domain invariant (i.e. content) and domain-variant (i.e. style) features of the music. Although there exists prior works which employ the Multi-modal Unsupervised Image-to-Image Translation (MUNIT) framework to perform the music style transfer in an unsupervised manner and successfully provide the promising results, the gap between the transferred music recordings and the real ones is still noticeable. In order to reduce such gap, we propose and experiment several techniques for improving the transferred results, including the domain balanced loss, up-sampling, content discriminator, recycle loss, and the data scaling. We conduct extensive experiments on the tasks of bilateral style transfer among four different genres, namely: piano solo, guitar solo, string quartet, and chiptune. In evaluation, an objective testing scheme is proposed to investigate the pros and cons of all our proposed techniques, while we also design a subjective testing method for making comparison among different approaches and show that our proposed method is able to provide superior performance with respect to the prior works.

原文English
主出版物標題Proceedings - 25th International Conference on Technologies and Applications of Artificial Intelligence, TAAI 2020
發行者Institute of Electrical and Electronics Engineers Inc.
頁面102-107
頁數6
ISBN(電子)9781665403801
DOIs
出版狀態Published - 12月 2020
事件25th International Conference on Technologies and Applications of Artificial Intelligence, TAAI 2020 - Taipei, Taiwan
持續時間: 3 12月 20205 12月 2020

出版系列

名字Proceedings - 25th International Conference on Technologies and Applications of Artificial Intelligence, TAAI 2020

Conference

Conference25th International Conference on Technologies and Applications of Artificial Intelligence, TAAI 2020
國家/地區Taiwan
城市Taipei
期間3/12/205/12/20

指紋

深入研究「Timbre-enhanced Multi-modal Music Style Transfer with Domain Balance Loss」主題。共同形成了獨特的指紋。

引用此