Hybrid accelerated optimization for speech recognition

Jen-Tzung Chien, Pei Wen Huang, Tan Lee

研究成果: Conference article同行評審

1 引文 斯高帕斯(Scopus)

摘要

Optimization procedure is crucial to achieve desirable performance for speech recognition based on deep neural networks (DNNs). Conventionally, DNNs are trained by using mini-batch stochastic gradient descent (SGD) which is stable but prone to be trapped into local optimum. A recent work based on Nesterov's accelerated gradient descent (NAG) algorithm is developed by merging the current momentum information into correction of SGD updating. NAG less likely jumps into local minimum so that convergence rate is improved. In general, optimization based on SGD is more stable while that based on NAG is faster and more accurate. This study aims to boost the performance of speech recognition by combining complimentary SGD and NAG. A new hybrid optimization is proposed by integrating the SGD with momentum and the NAG by using an interpolation scheme which is continuously run in each mini-batch according to the change rate of cost function in consecutive two learning epochs. Tradeoff between two algorithms can be balanced for mini-batch optimization. Experiments on speech recognition using CUSENT and Aurora-4 show the effectiveness of the hybrid accelerated optimization in DNN acoustic model.

原文English
頁(從 - 到)3399-3403
頁數5
期刊Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH
08-12-September-2016
DOIs
出版狀態Published - 1 一月 2016
事件17th Annual Conference of the International Speech Communication Association, INTERSPEECH 2016 - San Francisco, United States
持續時間: 8 九月 201616 九月 2016

指紋

深入研究「Hybrid accelerated optimization for speech recognition」主題。共同形成了獨特的指紋。

引用此