TY - GEN
T1 - A Dual-Critic Reinforcement Learning Framework for Frame-Level Bit Allocation in HEVC/H.265
AU - Ho, Yung Han
AU - Jin, Guo Lun
AU - Liang, Yun
AU - Peng, Wen-Hsiao
AU - Li, Xiaobo
N1 - Publisher Copyright:
© 2021 IEEE.
PY - 2021/3/23
Y1 - 2021/3/23
N2 - This paper introduces a dual-critic reinforcement learning (RL) framework to address the problem of frame-level bit allocation in HEVC/H.265. The objective is to minimize the distortion of a group of pictures (GOP) under a rate constraint. Previous RL-based methods tackle such a constrained optimization problem by maximizing a single reward function that often combines a distortion and a rate reward. However, the way how these rewards are combined is usually ad hoc and may not generalize well to various coding conditions and video sequences. To overcome this issue, we adapt the deep deterministic policy gradient (DDPG) reinforcement learning algorithm for use with two critics, with one learning to predict the distortion reward and the other the rate reward. In particular, the distortion critic works to update the agent when the rate constraint is satisfied. By contrast, the rate critic makes the rate constraint a priority when the agent goes over the bit budget. Experimental results on commonly used datasets show that our method outperforms the bit allocation scheme in x265 and the single-critic baseline by a significant margin in terms of rate-distortion performance while offering fairly precise rate control.
AB - This paper introduces a dual-critic reinforcement learning (RL) framework to address the problem of frame-level bit allocation in HEVC/H.265. The objective is to minimize the distortion of a group of pictures (GOP) under a rate constraint. Previous RL-based methods tackle such a constrained optimization problem by maximizing a single reward function that often combines a distortion and a rate reward. However, the way how these rewards are combined is usually ad hoc and may not generalize well to various coding conditions and video sequences. To overcome this issue, we adapt the deep deterministic policy gradient (DDPG) reinforcement learning algorithm for use with two critics, with one learning to predict the distortion reward and the other the rate reward. In particular, the distortion critic works to update the agent when the rate constraint is satisfied. By contrast, the rate critic makes the rate constraint a priority when the agent goes over the bit budget. Experimental results on commonly used datasets show that our method outperforms the bit allocation scheme in x265 and the single-critic baseline by a significant margin in terms of rate-distortion performance while offering fairly precise rate control.
KW - Dual critic
KW - Frame level bit allocation
KW - HEVC/H.265
KW - Reinforcement learning
UR - http://www.scopus.com/inward/record.url?scp=85105978370&partnerID=8YFLogxK
U2 - 10.1109/DCC50243.2021.00009
DO - 10.1109/DCC50243.2021.00009
M3 - Conference contribution
AN - SCOPUS:85105978370
T3 - Data Compression Conference Proceedings
SP - 213
EP - 222
BT - Proceedings - DCC 2021
A2 - Bilgin, Ali
A2 - Marcellin, Michael W.
A2 - Serra-Sagrista, Joan
A2 - Storer, James A.
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 2021 Data Compression Conference, DCC 2021
Y2 - 23 March 2021 through 26 March 2021
ER -