Facial Chirality: From Visual Self-Reflection to Robust Facial Feature Learning

Ling Lo, Hong Xia Xie, Hong Han Shuai, Wen Huang Cheng

Research output: Contribution to journalArticlepeer-review

4 Scopus citations

Abstract

As a fundamental vision task, facial expression recognition has made substantial progress recently. However, the recognition performance often degrades significantly in real-world scenarios due to the lack of robust facial features. In this paper, we propose an effective facial feature learning method that takes the advantage of facial chirality to discover the discriminative features for facial expression recognition. Most previous studies implicitly assume that human faces are symmetric. However, our work reveals that the facial asymmetric effect can be a crucial clue. Given a face image and its reflection without additional labels, we decouple the emotion-invariant facial features from the input image pair to better capture the emotion-related facial features. Moreover, as our model aligns emotion-related features of the image pair to enhance the recognition performance, the value of precise facial landmark alignment as a pre-processing step is reconsidered in this paper. Experiments demonstrate that the learned emotion-related features outperform the state of the art methods on several facial expression recognition benchmarks as well as real-world occlusion datasets, which manifests the effectiveness and robustness of the proposed model.

Original languageEnglish
Pages (from-to)1-11
Number of pages11
JournalIEEE Transactions on Multimedia
DOIs
StateAccepted/In press - 2022

Keywords

  • deep learning
  • Face recognition
  • Faces
  • facial expression
  • Facial features
  • feature disentanglement
  • Feature extraction
  • Reflection
  • Robustness
  • Transformers
  • vision transformer
  • visual chirality

Fingerprint

Dive into the research topics of 'Facial Chirality: From Visual Self-Reflection to Robust Facial Feature Learning'. Together they form a unique fingerprint.

Cite this