Hybrid accelerated optimization for speech recognition

Jen-Tzung Chien, Pei Wen Huang, Tan Lee

    研究成果: Conference article同行評審

    1 引文 斯高帕斯(Scopus)


    Optimization procedure is crucial to achieve desirable performance for speech recognition based on deep neural networks (DNNs). Conventionally, DNNs are trained by using mini-batch stochastic gradient descent (SGD) which is stable but prone to be trapped into local optimum. A recent work based on Nesterov's accelerated gradient descent (NAG) algorithm is developed by merging the current momentum information into correction of SGD updating. NAG less likely jumps into local minimum so that convergence rate is improved. In general, optimization based on SGD is more stable while that based on NAG is faster and more accurate. This study aims to boost the performance of speech recognition by combining complimentary SGD and NAG. A new hybrid optimization is proposed by integrating the SGD with momentum and the NAG by using an interpolation scheme which is continuously run in each mini-batch according to the change rate of cost function in consecutive two learning epochs. Tradeoff between two algorithms can be balanced for mini-batch optimization. Experiments on speech recognition using CUSENT and Aurora-4 show the effectiveness of the hybrid accelerated optimization in DNN acoustic model.

    頁(從 - 到)3399-3403
    期刊Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH
    出版狀態Published - 1 一月 2016
    事件17th Annual Conference of the International Speech Communication Association, INTERSPEECH 2016 - San Francisco, United States
    持續時間: 8 九月 201616 九月 2016


    深入研究「Hybrid accelerated optimization for speech recognition」主題。共同形成了獨特的指紋。