Maximum entropy learning with deep belief networks

Payton Lin, Szu Wei Fu, Syu Siang Wang, Ying Hui Lai, Yu Tsao*

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

11 Scopus citations

Abstract

Conventionally, the maximum likelihood (ML) criterion is applied to train a deep belief network (DBN). We present a maximum entropy (ME) learning algorithm for DBNs, designed specifically to handle limited training data. Maximizing only the entropy of parameters in the DBN allows more effective generalization capability, less bias towards data distributions, and robustness to over-fitting compared to ML learning. Results of text classification and object recognition tasks demonstrate ME-trained DBN outperforms ML-trained DBN when training data is limited.

Original languageEnglish
Article number251
JournalEntropy
Volume18
Issue number7
DOIs
StatePublished - Jul 2016

Keywords

  • Deep belief networks
  • Deep learning
  • Deep neural networks
  • Low-resource tasks
  • Machine learning
  • Maximum entropy
  • Restricted Boltzmann machine

Fingerprint

Dive into the research topics of 'Maximum entropy learning with deep belief networks'. Together they form a unique fingerprint.

Cite this