Text-to-Speech with Model Compression on Edge Devices

The application of voice services has become more common in daily life, including traffic navigation, voice assistants, audio books and so on. However, considering the cost and variability, it is difficult to fully utilize real voice recordings in different scenarios. In practice, speech synthesis technology is usually used to mimic human voices; On the other hand, with the development of computer equipment, the computing power of edge devices has also gradually improved, which enables light deep-learning network inference. Currently, many deep learning technologies have been ported to edge devices to create different applications, such as face recognition, speech recognition, and photo retouching. Therefore, if the speech synthesis network is ported to edge devices, with the advent of the fifth generation mobile communication generation (5G), it would be able to provide more innovative basis for voice services. In this research, the speech synthesis network Tacotron2 [1] + CBHG [2] will be ported to edge device and aims to optimize the model inference time and amount of parameters. The model optimization would be based on the compression of deep learning network, quantization, structured pruning and low-rank matrix approximation techniques to allow the speech synthesis network working effectively on edge devices. On the other hand, we get over the difference in library support between TensorFlow 1.5 and TensorFlow Lite. After the compression of the model, the inference speed of the Tacotron2 speech synthesis network on edge device is increased by 1.91 times, while the model size is reduced by 86% respectively.