Semantic segmentation on compressed video using block motion compensation and guided inpainting

Stefanie Tanujaya, Tieh Chu, Jia Hao Liu, Wen Hsiao Peng

研究成果: Conference contribution同行評審

4 引文 斯高帕斯(Scopus)

摘要

This paper addresses the problem of fast semantic segmentation on compressed video. Unlike most prior works for video segmentation, which perform feature propagation based on optical flow estimates or sophisticated warping techniques, ours takes advantage of block motion vectors in the compressed bitstream to propagate the segmentation of a keyframe to subsequent non-keyframes. This approach, however, needs to respect the inter-frame prediction structure, which often suggests recursive, multi-step prediction with error propagation and accumulation in the temporal dimension. To tackle the issue, we refine the motion-compensated segmentation using inpainting. Our inpainting network incorporates guided non-local attention for long-range reference and pixel-adaptive convolution for ensuring the local coherence of the segmentation. A fusion step then follows to combine both the motion-compensated and inpainted segmentations. Experimental results show that our method outperforms the state-of-the-art baselines in terms of segmentation accuracy. Moreover, it introduces the least amount of network parameters and multiply-add operations for non-keyframe segmentation.

原文English
主出版物標題2020 IEEE International Symposium on Circuits and Systems, ISCAS 2020 - Proceedings
發行者Institute of Electrical and Electronics Engineers Inc.
ISBN(電子)9781728133201
DOIs
出版狀態Published - 12 10月 2020
事件52nd IEEE International Symposium on Circuits and Systems, ISCAS 2020 - Virtual, Online
持續時間: 10 10月 202021 10月 2020

出版系列

名字Proceedings - IEEE International Symposium on Circuits and Systems
2020-October
ISSN(列印)0271-4310

Conference

Conference52nd IEEE International Symposium on Circuits and Systems, ISCAS 2020
城市Virtual, Online
期間10/10/2021/10/20

指紋

深入研究「Semantic segmentation on compressed video using block motion compensation and guided inpainting」主題。共同形成了獨特的指紋。

引用此