Hairstyle-and-identity-aware facial image style transfer with region-guiding masks

Hsin Ying Wang, Chiu Wei Chien, Ming Han Tsai*, I. Chen Lin*

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

Abstract

Face style transfer aims to transfer parts of appearance features from a reference to a source face and to keep the important characteristics of the source as well. During face style transfer, hairstyle structure of the reference and personal identity of the source are two important appearance characteristics, but few existing models can maintain both of these features during the transfer process simultaneously. To concurrently retain the two important features, we propose a framework with two stages to deal with the two sub-tasks respectively. In the first stage, our model uses region-guiding masks and the modified cycle loss to generate an image where the hairstyle is transferred adequately. Based on the hairstyle-transferred face, we focus on preserving the identity with a face replacement procedure and refining the image quality in the second stage. In addition to hair-style-aware transfer, we also exhibit that this framework can be extended for other region-aware tasks, such as eyeglasses transfer. User evaluation and the application of face recognition as a metric demonstrated that our proposed framework can generate high-quality images with accurate hairstyle transfer from the reference image while preserving the identity of the source image.

Original languageEnglish
JournalMultimedia Tools and Applications
DOIs
StateAccepted/In press - 2023

Keywords

  • Face identity preservation
  • Face style transfer
  • Hairstyle transfer
  • Regional masks

Fingerprint

Dive into the research topics of 'Hairstyle-and-identity-aware facial image style transfer with region-guiding masks'. Together they form a unique fingerprint.

Cite this