Neural Capacity Estimators: How Reliable Are They?

Farhad Mirkarimi, Stefano Rini, Nariman Farsad

研究成果: Conference contribution同行評審

摘要

Recently, several methods have been proposed for estimating the mutual information from sample data using deep neural networks and without the knowledge of closed form distribution of the data. This class of estimators is referred to as neural mutual information estimators. Although very promising, such techniques have yet to be rigorously bench-marked so as to establish their efficacy, ease of implementation, and stability for capacity estimation which is joint maximization frame-work. In this paper, we compare the different techniques proposed in the literature for estimating capacity and provide a practitioner perspective on their effectiveness. In particular, we study the performance of mutual information neural estimator (MINE), smoothed mutual information lower-bound estimator (SMILE), and directed information neural estimator (DINE) and provide insights on InfoNCE. We evaluated these algorithms in terms of their ability to learn the input distributions that are capacity-approaching for the AWGN channel, the optical intensity channel, and peak power-constrained AWGN channel. For both scenarios, we provide insightful comments on various aspects of the training process, such as accuracy, stability, and sensitivity to initialization.

原文English
主出版物標題ICC 2022 - IEEE International Conference on Communications
發行者Institute of Electrical and Electronics Engineers Inc.
頁面3868-3873
頁數6
ISBN(電子)9781538683477
DOIs
出版狀態Published - 2022
事件2022 IEEE International Conference on Communications, ICC 2022 - Seoul, Korea, Republic of
持續時間: 16 5月 202220 5月 2022

出版系列

名字IEEE International Conference on Communications
2022-May
ISSN(列印)1550-3607

Conference

Conference2022 IEEE International Conference on Communications, ICC 2022
國家/地區Korea, Republic of
城市Seoul
期間16/05/2220/05/22

指紋

深入研究「Neural Capacity Estimators: How Reliable Are They?」主題。共同形成了獨特的指紋。

引用此