An RNN-based noise estimation and likelihood compensation for noisy speech recognition

In this paper, a novel integration of RNN and PMC (parallel model combination) is presented for noisy speech recognition. It first employs an RNN to make the noise/speech discrimination. Then, by viewing the RNN outputs as the membership functions of noise and speech, an on-line noise tracking is performed for noise estimation. Also, a confidence measure is defined to represent the degree of the reliability of noise estimate and used to smooth the noise estimate across segments. The noise estimate is then used in PMC to adapt the HMM models trained from clean speech. Lastly, the RNN outputs are used to weight the likelihood scores of the PMC for softly reduce the influence of noise frame in the final decision. Experimental results showed that a significant improvement on recognition performance has been achieved under the non-stationary noise environment.