Collaborative Pseudo Labeling for Prompt-Based Learning

Prompt-based learning has been recently popular as an emerging data efficient approach to parameter fine-tuning for various few-shot classification tasks. The knowledge transfer from a large-scale pre-trained language model could be systematically performed. It is challenging to conduct transfer learning from unlabeled data. Pseudo labeling is feasible to enhance the model generalization by unsupervised learning where additional pseudo labeled data are augmented. However, this method likely suffers from the model bias problem due to the incorrect pseudo labels. This paper presents a new collaborative pseudo labeling for prompt learning which is implemented through a teacher-student model where teacher and student are learned collaboratively. Accordingly, the resulting pseudo labels do not mislead the model while the few-shot classification task can be improved. The experiments on few-shot sentiment classification and natural language inference show the merit of collaborative pseudo labeling and prompt tuning compared with the other methods.