FashionMirror: Co-attention Feature-remapping Virtual Try-on with Sequential Template Poses

Chieh Yun Chen, Ling Lo, Pin Jui Huang, Hong Han Shuai, Wen Huang Cheng

研究成果: Conference contribution同行評審

21 引文 斯高帕斯(Scopus)

摘要

Virtual try-on tasks have drawn increased attention. Prior arts focus on tackling this task via warping clothes and fusing the information at the pixel level with the help of semantic segmentation. However, conducting semantic segmentation is time-consuming and easily causes error accumulation over time. Besides, warping the information at the pixel level instead of the feature level limits the performance (e.g., unable to generate different views) and is unstable since it directly demonstrates the results even with a misalignment. In contrast, fusing information at the feature level can be further refined by the convolution to obtain the final results. Based on these assumptions, we propose a co-attention feature-remapping framework, namely FashionMirror, that generates the try-on results according to the driven-pose sequence in two stages. In the first stage, we consider the source human image and the target try-on clothes to predict the removed mask and the try-on clothing mask, which replaces the pre-processed semantic segmentation and reduces the inference time. In the second stage, we first remove the clothes on the source human via the removed mask and warp the clothing features conditioning on the try-on clothing mask to fit the next frame human. Meanwhile, we predict the optical flows from the consecutive 2D poses and warp the source human to the next frame at the feature level. Then, we enhance the clothing features and source human features in every frame to generate realistic try-on results with spatio-temporal smoothness. Both qualitative and quantitative results show that FashionMirror outperforms the state-of-the-art virtual try-on approaches.

原文English
主出版物標題Proceedings - 2021 IEEE/CVF International Conference on Computer Vision, ICCV 2021
發行者Institute of Electrical and Electronics Engineers Inc.
頁面13789-13798
頁數10
ISBN(電子)9781665428125
DOIs
出版狀態Published - 2021
事件18th IEEE/CVF International Conference on Computer Vision, ICCV 2021 - Virtual, Online, Canada
持續時間: 11 10月 202117 10月 2021

出版系列

名字Proceedings of the IEEE International Conference on Computer Vision
ISSN(列印)1550-5499

Conference

Conference18th IEEE/CVF International Conference on Computer Vision, ICCV 2021
國家/地區Canada
城市Virtual, Online
期間11/10/2117/10/21

指紋

深入研究「FashionMirror: Co-attention Feature-remapping Virtual Try-on with Sequential Template Poses」主題。共同形成了獨特的指紋。

引用此