Group-and-Conquer for Multi-Speaker Single-Channel Speech Separation

We propose to reduce the difficulty level by separating the separation process in a group-and-conquer way. Specifically, in the first stage, we propose a prediction model to estimate the optimal number of groups based on the input mixed signal. In addition, we train a group separation model to separate a mixed signal into multiple groups according to the number of groups. By training a vocal network with the triplet cosine loss and a group separation network simultaneously, the proposed group separation model better learns the latent feature of each group. As such, given the predicted number of groups, the group separation model can automatically separate the input audio signal into several groups. In the second stage, for the group with more than one speaker, the separation model focuses on fine-grained information to better separate the speech among the group. Experimental results show that our approach outperforms the state-of-the-art models by at least 8.68% in SI-SNRi.