RecFormer: personalized temporal-aware transformer for fair music recommendation

Recommendation systems have improved the characterization of user preferences by modeling their digital footprints and item content. However, another facet, model behavior, has attracted a great deal of attention in both academic and industry fields in recent years due to the increasing awareness of fairness. The shared task, a Rounded Evaluation of Recommender Systems (EvalRS @ CIKM 2022), is introduced to broadly measure multifaceted model predictions for music recommendation. To tackle the problem, we propose the RecFormer architecture with a personalized temporal-aware transformer to model the interactions among user history in a single framework. Specifically, RecFormer adopts a masked language modeling task as the training objective, which enables the model to capture fine-grained track embeddings by reconstructing tracks. Meanwhile, it also integrates a temporal-aware self-attention mechanism into the Transformer architecture so that the model is able to consider time-variant information among different users. Moreover, we introduce linearized attention to reduce quadratic computation and memory cost since the limited time is one of the challenges in this task. Extensive experiments and analysis are conducted to demonstrate the effectiveness of our RecFormer compared with the official baseline, and we examine the model contribution from the ablation study. Our team, yao0510, won the seventh prize with a total score of 0.1964 in the EvalRS challenge, which illustrates that our model achieved competitive performance. The source code will be publicly available at https://github.com/wywyWang/RecFormer.