Challenges and opportunities toward online training acceleration using RRAM-based hardware neural network

Chih Cheng Chang; Jen Chieh Liu; Yu Lin Shen; Teyuh Chou; Pin Chun Chen; I. Ting Wang; Chih Chun Su; Ming Hong Wu; Boris Hudec; Che Chia Chang; Chia-Ming Tsai; Tian-Sheuan Chang; H. S.Philip Wong; Tuo-Hung Hou

doi:10.1109/IEDM.2017.8268373

Challenges and opportunities toward online training acceleration using RRAM-based hardware neural network

Chih Cheng Chang, Jen Chieh Liu, Yu Lin Shen, Teyuh Chou, Pin Chun Chen, I. Ting Wang, Chih Chun Su, Ming Hong Wu, Boris Hudec, Che Chia Chang, Chia-Ming Tsai, Tian-Sheuan Chang, H. S.Philip Wong, Tuo-Hung Hou

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

19 Scopus citations

Abstract

This paper highlights the feasible routes of using resistive memory (RRAM) for accelerating online training of deep neural networks (DNNs). A high degree of asymmetric nordinearity in analog RRAMs could be tolerated when weight update algorithms are optimized with reduced training noise. Hybrid-weight Net (HW-Net), a modified multilayer perceptron (MLP) algorithm that utilizes hybrid internal analog and external binary weights is also proposed. Highly accurate online training could be realized using simple binary RRAMs that have already been widely developed as digital memory.

Original language	English
Title of host publication	2017 IEEE International Electron Devices Meeting, IEDM 2017
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	11.6.1-11.6.4
Number of pages	4
ISBN (Electronic)	9781538635599
DOIs	https://doi.org/10.1109/IEDM.2017.8268373
State	Published - 23 Jan 2018
Event	63rd IEEE International Electron Devices Meeting, IEDM 2017 - San Francisco, United States Duration: 2 Dec 2017 → 6 Dec 2017

Publication series

Name	Technical Digest - International Electron Devices Meeting, IEDM
ISSN (Print)	0163-1918

Conference

Conference	63rd IEEE International Electron Devices Meeting, IEDM 2017
Country/Territory	United States
City	San Francisco
Period	2/12/17 → 6/12/17

Access to Document

10.1109/IEDM.2017.8268373

Cite this

Chang, C. C., Liu, J. C., Shen, Y. L., Chou, T., Chen, P. C., Wang, I. T., Su, C. C., Wu, M. H., Hudec, B., Chang, C. C., Tsai, C.-M., Chang, T.-S., Wong, H. S. P., & Hou, T.-H. (2018). Challenges and opportunities toward online training acceleration using RRAM-based hardware neural network. In 2017 IEEE International Electron Devices Meeting, IEDM 2017 (pp. 11.6.1-11.6.4). (Technical Digest - International Electron Devices Meeting, IEDM). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/IEDM.2017.8268373

Chang, Chih Cheng ; Liu, Jen Chieh ; Shen, Yu Lin et al. / Challenges and opportunities toward online training acceleration using RRAM-based hardware neural network. 2017 IEEE International Electron Devices Meeting, IEDM 2017. Institute of Electrical and Electronics Engineers Inc., 2018. pp. 11.6.1-11.6.4 (Technical Digest - International Electron Devices Meeting, IEDM).

@inproceedings{e9f9ac58be794c3f9ff61fd21272921f,

title = "Challenges and opportunities toward online training acceleration using RRAM-based hardware neural network",

abstract = "This paper highlights the feasible routes of using resistive memory (RRAM) for accelerating online training of deep neural networks (DNNs). A high degree of asymmetric nordinearity in analog RRAMs could be tolerated when weight update algorithms are optimized with reduced training noise. Hybrid-weight Net (HW-Net), a modified multilayer perceptron (MLP) algorithm that utilizes hybrid internal analog and external binary weights is also proposed. Highly accurate online training could be realized using simple binary RRAMs that have already been widely developed as digital memory.",

author = "Chang, {Chih Cheng} and Liu, {Jen Chieh} and Shen, {Yu Lin} and Teyuh Chou and Chen, {Pin Chun} and Wang, {I. Ting} and Su, {Chih Chun} and Wu, {Ming Hong} and Boris Hudec and Chang, {Che Chia} and Chia-Ming Tsai and Tian-Sheuan Chang and Wong, {H. S.Philip} and Tuo-Hung Hou",

note = "Publisher Copyright: {\textcopyright} 2017 IEEE.; 63rd IEEE International Electron Devices Meeting, IEDM 2017 ; Conference date: 02-12-2017 Through 06-12-2017",

year = "2018",

month = jan,

day = "23",

doi = "10.1109/IEDM.2017.8268373",

language = "English",

series = "Technical Digest - International Electron Devices Meeting, IEDM",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "11.6.1--11.6.4",

booktitle = "2017 IEEE International Electron Devices Meeting, IEDM 2017",

address = "美國",

}

Chang, CC, Liu, JC, Shen, YL, Chou, T, Chen, PC, Wang, IT, Su, CC, Wu, MH, Hudec, B, Chang, CC, Tsai, C-M , Chang, T-S, Wong, HSP & Hou, T-H 2018, Challenges and opportunities toward online training acceleration using RRAM-based hardware neural network. in 2017 IEEE International Electron Devices Meeting, IEDM 2017. Technical Digest - International Electron Devices Meeting, IEDM, Institute of Electrical and Electronics Engineers Inc., pp. 11.6.1-11.6.4, 63rd IEEE International Electron Devices Meeting, IEDM 2017, San Francisco, United States, 2/12/17. https://doi.org/10.1109/IEDM.2017.8268373

Challenges and opportunities toward online training acceleration using RRAM-based hardware neural network. / Chang, Chih Cheng; Liu, Jen Chieh; Shen, Yu Lin et al.
2017 IEEE International Electron Devices Meeting, IEDM 2017. Institute of Electrical and Electronics Engineers Inc., 2018. p. 11.6.1-11.6.4 (Technical Digest - International Electron Devices Meeting, IEDM).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Challenges and opportunities toward online training acceleration using RRAM-based hardware neural network

AU - Chang, Chih Cheng

AU - Liu, Jen Chieh

AU - Shen, Yu Lin

AU - Chou, Teyuh

AU - Chen, Pin Chun

AU - Wang, I. Ting

AU - Su, Chih Chun

AU - Wu, Ming Hong

AU - Hudec, Boris

AU - Chang, Che Chia

AU - Tsai, Chia-Ming

AU - Chang, Tian-Sheuan

AU - Wong, H. S.Philip

AU - Hou, Tuo-Hung

PY - 2018/1/23

Y1 - 2018/1/23

N2 - This paper highlights the feasible routes of using resistive memory (RRAM) for accelerating online training of deep neural networks (DNNs). A high degree of asymmetric nordinearity in analog RRAMs could be tolerated when weight update algorithms are optimized with reduced training noise. Hybrid-weight Net (HW-Net), a modified multilayer perceptron (MLP) algorithm that utilizes hybrid internal analog and external binary weights is also proposed. Highly accurate online training could be realized using simple binary RRAMs that have already been widely developed as digital memory.

AB - This paper highlights the feasible routes of using resistive memory (RRAM) for accelerating online training of deep neural networks (DNNs). A high degree of asymmetric nordinearity in analog RRAMs could be tolerated when weight update algorithms are optimized with reduced training noise. Hybrid-weight Net (HW-Net), a modified multilayer perceptron (MLP) algorithm that utilizes hybrid internal analog and external binary weights is also proposed. Highly accurate online training could be realized using simple binary RRAMs that have already been widely developed as digital memory.

UR - http://www.scopus.com/inward/record.url?scp=85045211885&partnerID=8YFLogxK

U2 - 10.1109/IEDM.2017.8268373

DO - 10.1109/IEDM.2017.8268373

M3 - Conference contribution

AN - SCOPUS:85045211885

T3 - Technical Digest - International Electron Devices Meeting, IEDM

SP - 11.6.1-11.6.4

BT - 2017 IEEE International Electron Devices Meeting, IEDM 2017

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 63rd IEEE International Electron Devices Meeting, IEDM 2017

Y2 - 2 December 2017 through 6 December 2017

ER -

Chang CC, Liu JC, Shen YL, Chou T, Chen PC, Wang IT et al. Challenges and opportunities toward online training acceleration using RRAM-based hardware neural network. In 2017 IEEE International Electron Devices Meeting, IEDM 2017. Institute of Electrical and Electronics Engineers Inc. 2018. p. 11.6.1-11.6.4. (Technical Digest - International Electron Devices Meeting, IEDM). doi: 10.1109/IEDM.2017.8268373

Challenges and opportunities toward online training acceleration using RRAM-based hardware neural network

Abstract

Publication series

Conference

Access to Document

Other files and links

Fingerprint

Cite this