Taming NLU Noise: Student-Teacher Learning for Robust Dialogue Policy

Dialogue policy is a crucial component of dialogue systems, responsible for determining system responses based on user inputs. While reinforcement learning (RL) can effectively optimize the dialogue policy, the system performance in real-world settings is heavily influenced by an earlier component for natural language understanding (NLU). Once the NLU produces a wrong information, the dialogue policy will be affected to degrade the performance. To enhance the robustness of dialogue policy, this paper proposes integrating RL optimization with a noisy student-teacher learning, taming the noise generated by NLU. To prevent overconfidence during knowledge transfer from the teacher, we introduce a dual-teacher mechanism where knowledge distillation is carried out by using dynamic changes in the samples stored in the replay buffer which leverages the exploration-exploitation paradigm from RL. Evaluations on multi-domain multi-turn dialogue tasks demonstrate the effectiveness of this approach which shows the increased robustness to noisy NLU outputs and accordingly the improved overall system performance.