Mitigating Asymmetric Nonlinear Weight Update Effects in Hardware Neural Network Based on Analog Resistive Synapse

Chih Cheng Chang; Pin Chun Chen; Teyuh Chou; I. Ting Wang; Boris Hudec; Che Chia Chang; Chia-Ming Tsai; Tian-Sheuan Chang; Tuo-Hung Hou

doi:10.1109/JETCAS.2017.2771529

Mitigating Asymmetric Nonlinear Weight Update Effects in Hardware Neural Network Based on Analog Resistive Synapse

Chih Cheng Chang, Pin Chun Chen, Teyuh Chou, I. Ting Wang, Boris Hudec, Che Chia Chang, Chia-Ming Tsai, Tian-Sheuan Chang, Tuo-Hung Hou^*

^*Corresponding author for this work

Research output: Contribution to journal › Article › peer-review

67 Scopus citations

Abstract

Asymmetric nonlinear weight update is considered as one of the major obstacles for realizing hardware neural networks based on analog resistive synapses, because it significantly compromises the online training capability. This paper provides new solutions to this critical issue through co-optimization with the hardware-applicable deep-learning algorithms. New insights on engineering activation functions and a threshold weight update scheme effectively suppress the undesirable training noise induced by inaccurate weight update. We successfully trained a two-layer perceptron network online and improved the classification accuracy of MNIST handwritten digit data set to 87.8%/94.8% by using 6-/8-b analog synapses, respectively, with extremely high asymmetric nonlinearity.

Original language	English
Pages (from-to)	116-124
Number of pages	9
Journal	IEEE Journal on Emerging and Selected Topics in Circuits and Systems
Volume	8
Issue number	1
DOIs	https://doi.org/10.1109/JETCAS.2017.2771529
State	Published - 1 Mar 2018

Keywords

multilayer perceptron
Neuromorphic computing
RRAM
synapse

Access to Document

10.1109/JETCAS.2017.2771529

Cite this

@article{158470d58c6446e99a8df4f6c5cea1ff,

title = "Mitigating Asymmetric Nonlinear Weight Update Effects in Hardware Neural Network Based on Analog Resistive Synapse",

abstract = "Asymmetric nonlinear weight update is considered as one of the major obstacles for realizing hardware neural networks based on analog resistive synapses, because it significantly compromises the online training capability. This paper provides new solutions to this critical issue through co-optimization with the hardware-applicable deep-learning algorithms. New insights on engineering activation functions and a threshold weight update scheme effectively suppress the undesirable training noise induced by inaccurate weight update. We successfully trained a two-layer perceptron network online and improved the classification accuracy of MNIST handwritten digit data set to 87.8%/94.8% by using 6-/8-b analog synapses, respectively, with extremely high asymmetric nonlinearity.",

keywords = "multilayer perceptron, Neuromorphic computing, RRAM, synapse",

author = "Chang, {Chih Cheng} and Chen, {Pin Chun} and Teyuh Chou and Wang, {I. Ting} and Boris Hudec and Chang, {Che Chia} and Chia-Ming Tsai and Tian-Sheuan Chang and Tuo-Hung Hou",

year = "2018",

month = mar,

day = "1",

doi = "10.1109/JETCAS.2017.2771529",

language = "English",

volume = "8",

pages = "116--124",

journal = "IEEE Journal on Emerging and Selected Topics in Circuits and Systems",

issn = "2156-3357",

publisher = "IEEE Circuits and Systems Society",

number = "1",

}

TY - JOUR

T1 - Mitigating Asymmetric Nonlinear Weight Update Effects in Hardware Neural Network Based on Analog Resistive Synapse

AU - Chang, Chih Cheng

AU - Chen, Pin Chun

AU - Chou, Teyuh

AU - Wang, I. Ting

AU - Hudec, Boris

AU - Chang, Che Chia

AU - Tsai, Chia-Ming

AU - Chang, Tian-Sheuan

AU - Hou, Tuo-Hung

PY - 2018/3/1

Y1 - 2018/3/1

N2 - Asymmetric nonlinear weight update is considered as one of the major obstacles for realizing hardware neural networks based on analog resistive synapses, because it significantly compromises the online training capability. This paper provides new solutions to this critical issue through co-optimization with the hardware-applicable deep-learning algorithms. New insights on engineering activation functions and a threshold weight update scheme effectively suppress the undesirable training noise induced by inaccurate weight update. We successfully trained a two-layer perceptron network online and improved the classification accuracy of MNIST handwritten digit data set to 87.8%/94.8% by using 6-/8-b analog synapses, respectively, with extremely high asymmetric nonlinearity.

AB - Asymmetric nonlinear weight update is considered as one of the major obstacles for realizing hardware neural networks based on analog resistive synapses, because it significantly compromises the online training capability. This paper provides new solutions to this critical issue through co-optimization with the hardware-applicable deep-learning algorithms. New insights on engineering activation functions and a threshold weight update scheme effectively suppress the undesirable training noise induced by inaccurate weight update. We successfully trained a two-layer perceptron network online and improved the classification accuracy of MNIST handwritten digit data set to 87.8%/94.8% by using 6-/8-b analog synapses, respectively, with extremely high asymmetric nonlinearity.

KW - multilayer perceptron

KW - Neuromorphic computing

KW - RRAM

KW - synapse

UR - http://www.scopus.com/inward/record.url?scp=85033664458&partnerID=8YFLogxK

U2 - 10.1109/JETCAS.2017.2771529

DO - 10.1109/JETCAS.2017.2771529

M3 - Article

AN - SCOPUS:85033664458

SN - 2156-3357

VL - 8

SP - 116

EP - 124

JO - IEEE Journal on Emerging and Selected Topics in Circuits and Systems

JF - IEEE Journal on Emerging and Selected Topics in Circuits and Systems

IS - 1

ER -

Mitigating Asymmetric Nonlinear Weight Update Effects in Hardware Neural Network Based on Analog Resistive Synapse

Abstract

Keywords

Access to Document

Other files and links

Fingerprint

Cite this