Measuring Objective Visual Quality of Real-time Communication Systems in the Wild

Chih Fan Hsu, Chun Ying Huang, Xin Liu

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

During the outbreak of epidemic diseases, the demand for real-time communication (RTC) systems has dramatically increased. People use RTC systems to communicate with each other, attend online courses, present projects, and share videos. An open question, how to choose an appropriate system for high-quality communication with different network conditions and usage scenarios, comes RTC users. In this paper, we propose a scheme to systematically measure the visual quality of RTC systems. Our scheme does not need to modify systems and thus can be easily adapted to other RTC systems and video transmission systems, i.e., streaming systems. Based on this scheme, we measure six commercial RTC systems in four common usage scenarios. We measure the received video quality (graphical quality and frame rate) at the receiver and the upload bitrate at the sender. Furthermore, we propose a novel metric, area loss, to measure the system ability to handle insufficient bandwidth using video freezing and frame delay. Our detailed analysis reveals the advantages and disadvantages of each system. We expect the measurement scheme, the metric, and our findings can help the future development of RTC systems.

Original languageEnglish
Title of host publicationProceedings - 2021 IEEE 7th International Conference on Multimedia Big Data, BigMM 2021
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages9-16
Number of pages8
ISBN (Electronic)9781665434140
DOIs
StatePublished - 2021
Event7th IEEE International Conference on Multimedia Big Data, BigMM 2021 - Taichung, Taiwan
Duration: 15 Nov 202117 Nov 2021

Publication series

NameProceedings - 2021 IEEE 7th International Conference on Multimedia Big Data, BigMM 2021

Conference

Conference7th IEEE International Conference on Multimedia Big Data, BigMM 2021
Country/TerritoryTaiwan
CityTaichung
Period15/11/2117/11/21

Keywords

  • Performance analysis
  • Performance evaluation
  • Real-time communication systems

Fingerprint

Dive into the research topics of 'Measuring Objective Visual Quality of Real-time Communication Systems in the Wild'. Together they form a unique fingerprint.

Cite this