MS-SincResNet: Joint learning of 1D and 2D kernels using multi-scale SincNet and ResNet for music genre classification

Pei Chun Chang, Yong Sheng Chen, Chang Hsing Lee

研究成果: Conference contribution同行評審

2 引文 斯高帕斯(Scopus)

摘要

In this study, we proposed a new end-to-end convolutional neural network, called MS-SincResNet, for music genre classification. MS-SincResNet appends 1D multi-scale SincNet (MS-SincNet) to 2D ResNet as the first convolutional layer in an attempt to jointly learn 1D kernels and 2D kernels during the training stage. First, an input music signal is divided into a number of fixed-duration (3 seconds in this study) music clips, and the raw waveform of each music clip is fed into 1D MS-SincNet filter learning module to obtain three-channel 2D representations. The learned representations carry rich timbral, harmonic, and percussive characteristics comparing with spectrograms, harmonic spectrograms, percussive spectrograms and Mel-spectrograms. ResNet is then used to extract discriminative embeddings from these 2D representations. The spatial pyramid pooling (SPP) module is further used to enhance the feature discriminability, in terms of both time and frequency aspects, to obtain the classification label of each music clip. Finally, the voting strategy is applied to summarize the classification results from all 3-second music clips. In our experimental results, we demonstrate that the proposed MS-SincResNet outperforms the baseline SincNet and many well-known hand-crafted features. Considering individual 2D representation, MS-SincResNet also yields competitive results with the state-of-the-art methods on the GTZAN dataset and the ISMIR2004 dataset. The code is available at https://github.com/PeiChunChang/MS-SincResNet.

原文English
主出版物標題ICMR 2021 - Proceedings of the 2021 International Conference on Multimedia Retrieval
發行者Association for Computing Machinery, Inc
頁面29-36
頁數8
ISBN(電子)9781450384636
DOIs
出版狀態Published - 24 8月 2021
事件11th ACM International Conference on Multimedia Retrieval, ICMR 2021 - Taipei, Taiwan
持續時間: 16 11月 202119 11月 2021

出版系列

名字ICMR 2021 - Proceedings of the 2021 International Conference on Multimedia Retrieval

Conference

Conference11th ACM International Conference on Multimedia Retrieval, ICMR 2021
國家/地區Taiwan
城市Taipei
期間16/11/2119/11/21

指紋

深入研究「MS-SincResNet: Joint learning of 1D and 2D kernels using multi-scale SincNet and ResNet for music genre classification」主題。共同形成了獨特的指紋。

引用此