On Minimizing Analog Variation Errors to Resolve the Scalability Issue of ReRAM-Based Crossbar Accelerators

Yao Wen Kang, Chun Feng Wu, Yuan Hao Chang, Tei Wei Kuo*, Shu Yin Ho

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

10 Scopus citations

Abstract

Crossbar accelerators with a resistive random-access memory (ReRAM) are a promising solution for accelerating neural network applications. The advantages of achieving high computation throughput per watt make ReRAM-based crossbar accelerators become a potential solution for accelerating inference operations in the Internet of Things and edge devices. Due to the analog variation errors, the launched ReRAM-based crossbar accelerators can only perform well when each ReRAM cell is used to represent a limited number of data bits. To make such ReRAM-based crossbar accelerators applicable in wide application scenarios, several proposed researches target at binary neural networks and focus on the chip designs in relieving the implementation challenges on computation accuracy for realizing single-bit ReRAM-based crossbar accelerators. Even though several small-sized ReRAM-based crossbar accelerators are announced, the scalability issue hinders ReRAM-based crossbar accelerators from being scaled up. That is, when there are more and more wordline in an ReRAM-based crossbar accelerator, the analog variation error is amplified and thus seriously degrades the computation accuracy. In this work, we propose an adaptive data manipulation strategy to substantially reduce analog variation errors so as to fill up the gap on scaling up the ReRAM-based crossbar accelerators. In particular, a weight-rounding design is proposed to manipulate data to minimize overlapping variation so that the number of wordlines can be scaled up. In addition, an input subcycling design is proposed to further trade tolerable errors with neural networks' execution time. Moreover, a bitline redundant design is proposed to trade acceptable space overhead for eliminating the analog variation errors. The emulation experiments show that the proposed adaptive data manipulation strategy can improve the accuracy in running MNIST and CIFAR-10 by $1.3\times $ and $2.6\times $ with nearly no management penalty and hardware cost. The experimental results also show the close-to-ideal-case accuracy by substantially reducing analog variation errors.

Original languageEnglish
Article number9211412
Pages (from-to)3856-3867
Number of pages12
JournalIEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems
Volume39
Issue number11
DOIs
StatePublished - Nov 2020

Keywords

  • Accuracy
  • analog variation errors
  • crossbar accelerators
  • crossbar-based embedded systems
  • data manipulation
  • multiply accumulate (MAC)
  • neural networks
  • processing in memory (PIM)
  • resistive random-access memory (ReRAM)
  • scalability

Fingerprint

Dive into the research topics of 'On Minimizing Analog Variation Errors to Resolve the Scalability Issue of ReRAM-Based Crossbar Accelerators'. Together they form a unique fingerprint.

Cite this