Spatiotemporal super-resolution with cross-task consistency and its semi-supervised extension

Han Yi Lin, Pi Cheng Hsiu, Tei Wei Kuo, Yen Yu Lin

研究成果: Conference contribution同行評審

1 引文 斯高帕斯(Scopus)


Spatiotemporal super-resolution (SR) aims to upscale both the spatial and temporal dimensions of input videos, and produces videos with higher frame resolutions and rates. It involves two essential sub-tasks: spatial SR and temporal SR. We design a two-stream network for spatiotemporal SR in this work. One stream contains a temporal SR module followed by a spatial SR module, while the other stream has the same two modules in the reverse order. Based on the interchangeability of performing the two sub-tasks, the two network streams are supposed to produce consistent spatiotemporal SR results. Thus, we present a cross-stream consistency to enforce the similarity between the outputs of the two streams. In this way, the training of the two streams is correlated, which allows the two SR modules to share their supervisory signals and improve each other. In addition, the proposed cross-stream consistency does not consume labeled training data and can guide network training in an unsupervised manner. We leverage this property to carry out semi-supervised spatiotemporal SR. It turns out that our method makes the most of training data, and can derive an effective model with few high-resolution and high-frame-rate videos, achieving the state-of-the-art performance. The source code of this work is available at

主出版物標題Proceedings of the 29th International Joint Conference on Artificial Intelligence, IJCAI 2020
編輯Christian Bessiere
發行者International Joint Conferences on Artificial Intelligence
出版狀態Published - 2020
事件29th International Joint Conference on Artificial Intelligence, IJCAI 2020 - Yokohama, Japan
持續時間: 1 1月 2021 → …


名字IJCAI International Joint Conference on Artificial Intelligence


Conference29th International Joint Conference on Artificial Intelligence, IJCAI 2020
期間1/01/21 → …