Towards ultra-low-bitrate video conferencing using facial landmarks

Pin Chun Wang, Ching Ling Fan, Chun-Ying Huang, Kuan Ta Chen, Cheng Hsin Hsu

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

2 Scopus citations


Providing high-quality video conferencing experience over the best-effort Internet and wireless networks is challenging, because 2D videos are bulky. In this paper, we exploit the common structure of conferencing videos for an ultra-low-bitrate video conferencing system. In particular, we design, implement, optimize, and evaluate a video conferencing system, which: (i) extracts facial landmarks, (ii) transmits the selected facial landmarks and 2D images, and (iii) warps the untransmitted 2D images at the receiver. Several optimization techniques are adopted for minimizing the running time and maximizing the video quality, e.g., the image and warping frames are optimally determined based on network conditions and video content. The experiment results from real conferencing videos reveal that our proposed system: (i) outperforms the stateof-the-art x265 by up to 11.05 dB in PSNR (Peak Signal-to-Noise Ratio), (ii) adapts to different video content and network conditions, and (iii) runs in real-time at about 12 frame-per-second.

Original languageEnglish
Title of host publicationMM 2016 - Proceedings of the 2016 ACM Multimedia Conference
PublisherAssociation for Computing Machinery, Inc
Number of pages5
ISBN (Electronic)9781450336031
StatePublished - 1 Oct 2016
Event24th ACM Multimedia Conference, MM 2016 - Amsterdam, United Kingdom
Duration: 15 Oct 201619 Oct 2016

Publication series

NameMM 2016 - Proceedings of the 2016 ACM Multimedia Conference


Conference24th ACM Multimedia Conference, MM 2016
Country/TerritoryUnited Kingdom


  • Data compression
  • Facial landmarks
  • Facial models
  • Optimization
  • Video codecs
  • Video conferencing


Dive into the research topics of 'Towards ultra-low-bitrate video conferencing using facial landmarks'. Together they form a unique fingerprint.

Cite this