SME-net: Sparse motion estimation for parametric video prediction through reinforcement learning

Yung Han Ho*, Chuan Yuan Cho, Guo Lun Jin, Wen-Hsiao Peng

*此作品的通信作者

研究成果: Conference contribution同行評審

9 引文 斯高帕斯(Scopus)

摘要

This paper leverages a classic prediction technique, known as parametric overlapped block motion compensation (POBMC), in a reinforcement learning framework for video prediction. Learning-based prediction methods with explicit motion models often suffer from having to estimate large numbers of motion parameters with artificial regularization. Inspired by the success of sparse motion-based prediction for video compression, we propose a parametric video prediction on a sparse motion field composed of few critical pixels and their motion vectors. The prediction is achieved by gradually refining the estimate of a future frame in iterative, discrete steps. Along the way, the identification of critical pixels and their motion estimation are addressed by two neural networks trained under a reinforcement learning setting. Our model achieves the state-of-the-art performance on CaltchPed, UCF101 and CIF datasets in one-step and multi-step prediction tests. It shows good generalization results and is able to learn well on small training data.

原文English
主出版物標題Proceedings - 2019 International Conference on Computer Vision, ICCV 2019
發行者Institute of Electrical and Electronics Engineers Inc.
頁面10461-10469
頁數9
ISBN(電子)9781728148038
DOIs
出版狀態Published - 10月 2019
事件17th IEEE/CVF International Conference on Computer Vision, ICCV 2019 - Seoul, Korea, Republic of
持續時間: 27 10月 20192 11月 2019

出版系列

名字Proceedings of the IEEE International Conference on Computer Vision
2019-October
ISSN(列印)1550-5499

Conference

Conference17th IEEE/CVF International Conference on Computer Vision, ICCV 2019
國家/地區Korea, Republic of
城市Seoul
期間27/10/192/11/19

指紋

深入研究「SME-net: Sparse motion estimation for parametric video prediction through reinforcement learning」主題。共同形成了獨特的指紋。

引用此