Hairstyle-and-identity-aware facial image style transfer with region-guiding masks

Face style transfer aims to transfer parts of appearance features from a reference to a source face and to keep the important characteristics of the source as well. During face style transfer, hairstyle structure of the reference and personal identity of the source are two important appearance characteristics, but few existing models can maintain both of these features during the transfer process simultaneously. To concurrently retain the two important features, we propose a framework with two stages to deal with the two sub-tasks respectively. In the first stage, our model uses region-guiding masks and the modified cycle loss to generate an image where the hairstyle is transferred adequately. Based on the hairstyle-transferred face, we focus on preserving the identity with a face replacement procedure and refining the image quality in the second stage. In addition to hair-style-aware transfer, we also exhibit that this framework can be extended for other region-aware tasks, such as eyeglasses transfer. User evaluation and the application of face recognition as a metric demonstrated that our proposed framework can generate high-quality images with accurate hairstyle transfer from the reference image while preserving the identity of the source image.