Neural Capacity Estimators: How Reliable Are They?

Farhad Mirkarimi, Stefano Rini, Nariman Farsad

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

3 Scopus citations

Abstract

Recently, several methods have been proposed for estimating the mutual information from sample data using deep neural networks and without the knowledge of closed form distribution of the data. This class of estimators is referred to as neural mutual information estimators. Although very promising, such techniques have yet to be rigorously bench-marked so as to establish their efficacy, ease of implementation, and stability for capacity estimation which is joint maximization frame-work. In this paper, we compare the different techniques proposed in the literature for estimating capacity and provide a practitioner perspective on their effectiveness. In particular, we study the performance of mutual information neural estimator (MINE), smoothed mutual information lower-bound estimator (SMILE), and directed information neural estimator (DINE) and provide insights on InfoNCE. We evaluated these algorithms in terms of their ability to learn the input distributions that are capacity-approaching for the AWGN channel, the optical intensity channel, and peak power-constrained AWGN channel. For both scenarios, we provide insightful comments on various aspects of the training process, such as accuracy, stability, and sensitivity to initialization.

Original languageEnglish
Title of host publicationICC 2022 - IEEE International Conference on Communications
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages3868-3873
Number of pages6
ISBN (Electronic)9781538683477
DOIs
StatePublished - 2022
Event2022 IEEE International Conference on Communications, ICC 2022 - Seoul, Korea, Republic of
Duration: 16 May 202220 May 2022

Publication series

NameIEEE International Conference on Communications
Volume2022-May
ISSN (Print)1550-3607

Conference

Conference2022 IEEE International Conference on Communications, ICC 2022
Country/TerritoryKorea, Republic of
CitySeoul
Period16/05/2220/05/22

Keywords

  • AWGN channel
  • Neural capacity estimators
  • optimal input distribution
  • peak power-constrained AWGN channel
  • the optical intensity channel

Fingerprint

Dive into the research topics of 'Neural Capacity Estimators: How Reliable Are They?'. Together they form a unique fingerprint.

Cite this