TY - GEN
T1 - Hybrid Binary-Stochastic Computing-based ANN Design with Binary-in-Series-out ReLU
AU - Chen, Kun Chih Jimmy
AU - Chen, Cheng Ting
N1 - Publisher Copyright:
© 2022 IEEE.
PY - 2022
Y1 - 2022
N2 - Because of the remarkable performance on the problem of classification and recognition, Artificial Neural Networks (ANNs) have received much attention in recent years. However, due to the massive parameters and complicated interconnections between each neuron, ANN hardware architectures suffer from considerable area overhead and power consumption, which becomes the design challenge nowadays. To mitigate the design problem, the Stochastic Computing (SC)-based computing method has been proposed to use stochastic bit-stream for each kind of arithmetic operation in recent years. Besides, many SC-based ANN designs were proposed and showed their efficiency to save the computing power and area of the hardware. However, the conventional SC-based ANNs (SC-ANNs) suffer from low computing accuracy due to the applied stochastic bit-stream. To further improve the computing accuracy of the SC-ANN, we propose to replace the SC-based multiply-accumulator (MAC) with the parallel counter (PC) in this work. In addition, we further propose a binary-input-series-output (BISO) ReLU function to transfer the binary to the corresponding stochastic bit-stream efficiently. Compared with the conventional SC-ANN approach, the proposed SC-ANN design with PC-based MAC and BISO ReLU can improve computing accuracy by 86.6%. In addition, the proposed approach can reduce 95.3% area cost and 90% power consumption over than non-SC-ANN design, which achieves higher hardware efficiency.
AB - Because of the remarkable performance on the problem of classification and recognition, Artificial Neural Networks (ANNs) have received much attention in recent years. However, due to the massive parameters and complicated interconnections between each neuron, ANN hardware architectures suffer from considerable area overhead and power consumption, which becomes the design challenge nowadays. To mitigate the design problem, the Stochastic Computing (SC)-based computing method has been proposed to use stochastic bit-stream for each kind of arithmetic operation in recent years. Besides, many SC-based ANN designs were proposed and showed their efficiency to save the computing power and area of the hardware. However, the conventional SC-based ANNs (SC-ANNs) suffer from low computing accuracy due to the applied stochastic bit-stream. To further improve the computing accuracy of the SC-ANN, we propose to replace the SC-based multiply-accumulator (MAC) with the parallel counter (PC) in this work. In addition, we further propose a binary-input-series-output (BISO) ReLU function to transfer the binary to the corresponding stochastic bit-stream efficiently. Compared with the conventional SC-ANN approach, the proposed SC-ANN design with PC-based MAC and BISO ReLU can improve computing accuracy by 86.6%. In addition, the proposed approach can reduce 95.3% area cost and 90% power consumption over than non-SC-ANN design, which achieves higher hardware efficiency.
KW - artificial neural network
KW - machine learning
KW - parallel counter
KW - ReLU
KW - stochastic computing
UR - http://www.scopus.com/inward/record.url?scp=85139010210&partnerID=8YFLogxK
U2 - 10.1109/AICAS54282.2022.9869930
DO - 10.1109/AICAS54282.2022.9869930
M3 - Conference contribution
AN - SCOPUS:85139010210
T3 - Proceeding - IEEE International Conference on Artificial Intelligence Circuits and Systems, AICAS 2022
SP - 162
EP - 165
BT - Proceeding - IEEE International Conference on Artificial Intelligence Circuits and Systems, AICAS 2022
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 4th IEEE International Conference on Artificial Intelligence Circuits and Systems, AICAS 2022
Y2 - 13 June 2022 through 15 June 2022
ER -